Content modifier processor clears OTLP metadata

[nbroyles]

Bug Report

Describe the bug

When using a content modifier processor with OTLP data in the input or output, the content modifier clears/nukes the OTLP metadata associated with the log data. Additionally, the action to be taken by the content modifier fails (e.g. upsert key: value)

To Reproduce

• Use this config:

pipeline:
  inputs:
      - name: opentelemetry
        listen: 0.0.0.0
        port: 4318
        processors:
          logs:
              - name: content_modifier
                context: otel_resource_attributes
                action: upsert
                key: foo
                value: bar
  outputs:
      - name: stdout
        match: '*'

• (Optional, can just directly use the config above) Spin up a docker container:

docker run -ti   -v <path-to-config>:/fluent-bit/etc/config.yaml:ro  -p 4318:4318 -p 2020:2020 fluent/fluent-bit:3.2.9   -c /fluent-bit/etc/config.yaml

• Curl with log.json

curl -X POST http://localhost:4318/v1/logs -H "Content-Type: application/json" -d @log.json

Expected behavior

Metadata and attributes should be persisted like so (which happens if you remove the content-modifier processor from the listed config above). Also, manually updated the resource attributes to include foo: bar which is also expected.

[2025/03/18 22:07:02] [ info] [sp] stream processor started
[2025/03/18 22:07:02] [ info] [output:stdout:stdout.0] worker #0 started
GROUP METADATA :

{"schema"=>"otlp", "resource_id"=>0, "scope_id"=>0}

GROUP ATTRIBUTES :

{"resource"=>{"attributes"=>{"service.name"=>"my.service", "foo" => "bar"}}, "schema_url"=>"", "scope"=>{"name"=>"my.library", "version"=>"1.0.0", "attributes"=>{"my.scope.attribute"=>"some scope attribute"}}}

[0] v1_logs: [[1544712660.18446744071984150828, {"otlp"=>{"observed_timestamp"=>1544712660300000000, "timestamp"=>1544712660300000000, "severity_number"=>10, "severity_text"=>"Information", "attributes"=>{"string.attribute"=>"some string", "boolean.attribute"=>true, "int.attribute"=>10, "double.attribute"=>637.704000, "array.attribute"=>["many", "values"], "map.attribute"=>{"some.map.key"=>"some value"}}, "trace_id"=>"\xe4\x1f\x04\x14Q{\xf7\xcd7\xf3]7\x0fn\xbd\x07\xad\xf7\xf3]\xc5\x0b\xad\x02", "span_id"=>"\x10A5\xf4\x1e\xc4\x0bp\xb5\x07^\xf8", "trace_flags"=>0}}], {"message"=>"Example log record"}]

Actual behavior

Output will be the following (notice the missing metadata + attributes, and lack of a 'foo' field):

[2025/03/18 22:05:09] [ info] [sp] stream processor started
[2025/03/18 22:05:09] [ info] [output:stdout:stdout.0] worker #0 started
[0] v1_logs: [[1544712660.18446744071984150828, {"otlp"=>{"observed_timestamp"=>1544712660300000000, "timestamp"=>1544712660300000000, "severity_number"=>10, "severity_text"=>"Information", "attributes"=>{"string.attribute"=>"some string", "boolean.attribute"=>true, "int.attribute"=>10, "double.attribute"=>637.704000, "array.attribute"=>["many", "values"], "map.attribute"=>{"some.map.key"=>"some value"}}, "trace_id"=>"\xe4\x1f\x04\x14Q{\xf7\xcd7\xf3]7\x0fn\xbd\x07\xad\xf7\xf3]\xc5\x0b\xad\x02", "span_id"=>"\x10A5\xf4\x1e\xc4\x0bp\xb5\x07^\xf8", "trace_flags"=>0}}], {"message"=>"Example log record"}]

Screenshots

Your Environment

• Version used: FB 3.2.9
• Configuration: See the config above
• Environment name and version (e.g. Kubernetes? What version?):
• Server type and version:
• Operating System and version: Mac OS
• Filters and plugins:

Additional context

[edsiper]

@nbroyles thanks for bringing this up.

This is the current solution: #10117