[8.13](backport #4371) [Kubernetes secret provider] Send signal to agent if cache is updated

[mergify]

What does this PR do?

If the secrets in the cache get updated, then we notify the agent.

Why is it important?

See issue #4168 and the comments there.

Checklist

• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have made corresponding change to the default configuration files
• I have added tests that prove my fix is effective or that my feature works
• I have added an entry in ./changelog/fragments using the changelog tool
• I have added an integration test or an E2E test

How to test this PR locally

Check the README file in the root directory to see how to build your own custom image and loaded to kind cluster.

Example secret.

apiVersion: v1
kind: Secret
metadata:
  name: my-secret
data:
  extra: bXktc2VjcmV0LW1lc3NhZ2UK

Example input that you can add to EA standalone using the secret.

      - id: kubernetes-node-metrics
        type: kubernetes/metrics
        use_output: default
        meta:
          package:
            name: kubernetes
            version: 1.52.0
        data_stream:
          namespace: default
        streams:
          - data_stream:
              dataset: kubernetes.proxy
              type: metrics
            metricsets:
              - proxy
            hosts:
              - 'localhost:10249'
            period: 10s
            processors:
              - add_fields:
                  target: myfield
                  fields:
                    value: ${kubernetes_secrets.default.my-secret.extra}

Related issues

• Closes The kuberenetes secrets provider should regenerate the agent configuration when a secrets change is detected #4168

Screenshots

Using the setup as in the section on how to test locally, we can check that the new field has the correct value associated with the secret:

Logs

If a secret gets updated, then the cache will be updated and this wall cause the signal to get triggered. Logs:

{"log.level":"info","@timestamp":"2024-03-07T12:07:46.095Z","log.logger":"composable","log.origin":{"file.name":"kubernetessecrets/kubernetes_secrets.go","file.line":119},"message":"Secrets cache was updated, the agent will be notified.","log":{"source":"elastic-agent"},"ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2024-03-07T12:07:46.273Z","log.origin":{"file.name":"coordinator/coordinator.go","file.line":1382},"message":"component model updated","log":{"source":"elastic-agent"},"changes":{"components":{"updated":["kubernetes/metrics-default: [(kubernetes/metrics-default-kubernetes-node-metrics: updated)]"],"count":4},"outputs":{}},"ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2024-03-07T12:07:46.273Z","log.origin":{"file.name":"coordinator/coordinator.go","file.line":1181},"message":"Updating running component model","log":{"source":"elastic-agent"},"ecs.version":"1.6.0"}

---

This is an automatic backport of pull request #4371 done by [Mergify](https://mergify.com).

[cmacknz]

Needs #4368 first

[elastic-sonarqube]

Quality Gate passed

Kudos, no new issues were introduced!

0 New issues
0 Security Hotspots
100.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube