processor: add conditional processing support for logs

[niedbalski]

Processor conditional processing for logs

Fix #10127

This PR adds conditional processing support for logs in processors. This feature allows processors to conditionally process logs based on field values.

Example configurations

1. Simple

service:
  log_level: info
pipeline:
  inputs:
    - name: dummy
      dummy: '{"request": {"method": "GET", "path": "/api/v1/resource", "headers": {"Authorization": "Bearer valid-token"}, "access": "granted"}}'
      tag: error.msg
      processors:
        logs:
          - name: content_modifier
            action: insert
            key: modified_if_post
            value: true
            condition:
              op: and
              rules:
                - field: "$request['method']"
                  op: eq
                  value: "POST"

          - name: content_modifier
            action: insert
            key: modified_if_get
            value: true
            condition:
              op: and
              rules:
                - field: "$request['method']"
                  op: eq
                  value: "GET"

  outputs:
    - name: stdout
      match: '*'

When this configuration runs, the processor only adds the field modified_if_get to records where the request method is GET:

[0] error.msg: [[1743160978.853138000, {}], {"request"=>{"method"=>"GET", "path"=>"/api/v1/resource", "headers"=>{"Authorization"=>"Bearer valid-token"}, "access"=>"granted"}, "modified_if_get"=>"true"}]
[0] error.msg: [[1743160979.853940000, {}], {"request"=>{"method"=>"GET", "path"=>"/api/v1/resource", "headers"=>{"Authorization"=>"Bearer valid-token"}, "access"=>"granted"}, "modified_if_get"=>"true"}]

2. More complex

service:
  log_level: info
pipeline:
  inputs:
    - name: dummy
      dummy: '{"request": {"method": "GET", "path": "/api/v1/resource", "headers": {"Authorization": "Bearer valid-token", "Content-Type": "application/json"}, "status_code": 200, "response_time": 150}}'
      tag: request.log
      processors:
        logs:
          - name: content_modifier
            action: insert
            key: high_priority_method
            value: true
            condition:
              op: and
              rules:
                - field: "$request['method']"
                  op: in
                  value: ["POST", "PUT", "DELETE"]

          - name: content_modifier
            action: insert
            key: requires_performance_check
            value: true
            condition:
              op: or
              rules:
                - field: "$request['response_time']"
                  op: gt
                  value: 100
                - field: "$request['status_code']"
                  op: gte
                  value: 400

  outputs:
    - name: stdout

Outputs:

  "headers"=>{"Authorization"=>"Bearer valid-token", "Content-Type"=>"application/json"}, "status_code"=>200, "response_time"=>150},
  "requires_performance_check"=>"true"}]
  [0] request.log: [[1743163088.501242000, {}], {"request"=>{"method"=>"GET", "path"=>"/api/v1/resource", "headers"=>{"Authorization"=>"Bearer valid-token",
  "Content-Type"=>"application/json"}, "status_code"=>200, "response_time"=>150}, "requires_performance_check"=>"true"}]
  [0] request.log: [[1743163089.501156000, {}], {"request"=>{"method"=>"GET", "path"=>"/api/v1/resource", "headers"=>{"Authorization"=>"Bearer valid-token",
  "Content-Type"=>"application/json"}, "status_code"=>200, "response_time"=>150}, "requires_performance_check"=>"true"}]

1. The high_priority_method field was NOT added to the records because our input has "method":"GET" which isn't in the array ["POST", "PUT", "DELETE"] specified
   in the first condition.

2. The requires_performance_check field WAS added with value "true" because the second condition is met:

   • We have an OR condition with two rules
   • The first rule checks if response_time > 100 (our value is 150)
   • Since this rule is true, the whole condition evaluates to true, regardless of the second rule
   • This matches what we expected: when response time > 100, flag the request for performance checking

Added tests

Internal unit tests

• Added test suite for processor condition validation in tests/internal/processor_conditional.c
• Created tests for condition operators (and, or)
• Added tests for rule operators (eq, neq, gt, lt, gte, lte, regex, not_regex, in, not_in)
• Included tests for fields with $ prefix and record accessors
• Added tests for error cases:
  • Invalid operator validation
  • Empty rules arrays
  • Multiple rules handling
  • Context metadata handling
  • Deeply nested field access
  • Overwriting existing conditions
  • Missing fields/operators/values
  • Invalid rule structures
  • Invalid regex patterns
  • Array values for numeric operators
  • String values for 'in' operator
  • Complex nested conditions

Runtime shell tests

• Added runtime shell tests in tests/runtime_shell/processor_conditional.sh
• Tests processor condition functionality with actual Fluent Bit instances