feat(RFC): Add Internal Trace Model.

[hdost]

Provides a path forward for integrating sources and sinks for Tracing.

Relates #1444, #17307, #17308

[jszwedko]

Thanks for this @hdost . We'll try to review it next week in more detail.

[bruceg]

The remaining mappings of what?

[jszwedko]

I think we need both directions, no? The datadog_agent source needs to decode it and the datadog_traces sink needs to encode it.

[bruceg]

We discussed this internally and agree with the general direction of this proposal. It makes sense to adopt the the OpenTelemetry data types as the internal model, since the OTel project has defined many mappings to and from their data model for other trace types. So it stands as a useful intermediate representation for many kinds of traces.

However, if we are going to adopt that data model, we would prefer it be implemented in a more structured form. Specifically, we would want all of the data structures, were possible, to become fixed Rust structs instead of a generic map type. This would echo our metric types, which can represent a variety of different kinds of metrics with a fixed structure. This will simplify all kinds of code managing conversion to and from other data types, such as protobufs or other trace models, since they can rely on the presence of certain fields at a source code level.

As such, we would like to see the above reflected in the RFC. It should contain source code specifics of what the structures would look like and, if appropriate, what methods would be made available for manipulating them. Since this isn't a completely different direction, we don't think it requires a complete overhaul of this document but it does require changing the core data types, which means that consideration is very much in scope.

Some examples from existing RFCs to show what that might look like:

• https://github.com/vectordotdev/vector/blob/master/rfcs/2022-10-12-14742-flexible-metric-tags.md
• https://github.com/vectordotdev/vector/blob/master/rfcs/2022-07-28-13691-registered-internal-events.md
• https://github.com/vectordotdev/vector/blob/master/rfcs/2022-02-23-9481-config-schema.md

Many other RFCs contain Rust code samples, as we intend for these documents to be very opinionated as to the target solution, with many of the implications worked out.

[hdost]

Thank you for the thorough review, I'll take some time to reformulate with all that in mind.

[zsbfg]

@hdost - Just curious - has this PR been abandoned due to time constraints or did you move to another technology solution and no longer need to move forward with this PR in vector?

[hdost]

@hdost - Just curious - has this PR been abandoned due to time constraints or did you move to another technology solution and no longer need to move forward with this PR in vector?

Time constraints I'm afraid, but I should be spending some time soon again. However, don't be afraid to continue to patch it if you wish. I can always try to pick up. Just tag me on your pr

[zsbfg]

Thanks @hdost - I was hoping (selfishly) that you were going to be able work on this soon. I very much appreciate your time and attention.

[hdost]

@jszwedko I made some significant updates. I want to make sure it's going in the right direction as mentioned before. I apologize for the long delay.