feat(load-generator): add spans for feature flag evaluation

[aepfli]

Changes

As the span was created before the feature flag evaluation, we are now creating a span for each task, so we can enhance it with information when needed.

Merge Requirements

For new features contributions, please make sure you have completed the following
essential items:

• CHANGELOG.md updated to document new feature additions
• Appropriate documentation updates in the docs
• Appropriate Helm chart updates in the helm-charts

Maintainers will not merge until the above have been completed. If you're unsure
which docs need to be changed ping the
@open-telemetry/demo-approvers.

[julianocosta89]

I don't understand this PR.

The client.get already have an span being created automatically:

opentelemetry.instrumentation.requests instruments the outgoing calls.
My point on #2114 (comment) is that this is not linked at all with flagd.

[aepfli]

yes the client get does that, but when we "flood home" we do not have the span created before the feature flag evaluation starts, with this we have now a span with the flag evaluation result added. (i added the spans for the others only for consistency.

https://github.com/open-telemetry/opentelemetry-demo/pull/2119/files#diff-fc6de368e95786344dbd26e3e4b680a0cc6925ba9478b61ff9a6d9c7d107ae69R181-R185

The post will never be really linked to the span, because this is the OFREP protocol. The provider periodically fetched flag changes and stores them. The evaluation it self only uses the store information. there is no relation between flag configuration fetches, and flag evaluations. Now the flag result is added to the span of the load generation. (i agree that adding it for the others is maybe overkill)

see and the flag evaluation event added to the span

[aepfli]

i think we might have been talking about different things. i thought it is important to add the evaluation event to the span of the "flood home" action - which is actually the important information of the evaluation.

Fetching the flags used to be a call all the time with the rpc mode. but with the ofrep mode, we are fetching a generate flag configuration regularly, and have this configuration stored locally, and never doing a call for a flag evaluation to the host system. We could eg. only poll for this changes every x seconds, currently it is every 5 seconds.

[julianocosta89]

@mviitane and @puckpuck I'd like you opinion here.
I just had a chat with @aepfli and my initial concern is actually being tacked in this PR: open-feature/flagd#1593

This one here add spans to each call of the load-gen.
On one hand, this is nice, because it gives us span events on feature flag evaluation, but on the other hand, it bloats the number of spans produced by load-gen.

In a real life scenario, I'd love to be able to see all feature flag evaluation during my user journey, but I'm not sure if we want that for load-generator.

@aepfli would be fine in closing this issue, and I'm already happy with the flagd one I've mentioned above.

What do you think?

[puckpuck]

I don't think we need a span for each loadgen flag evaluation. We can kill close this PR.