Merge branch 'main' into add-otel-resource-detection-processor

Author: andrzej-stencel

.github/workflows/bump-golang.yml

@@ -22,7 +22,7 @@ jobs:
       - uses: actions/checkout@v4
 
       - name: Install Updatecli in the runner
-        uses: updatecli/updatecli-action@9a37c7e35598d7b37d8e7568b40ed9538112be01 # v0.76.1
+        uses: updatecli/updatecli-action@fa41baa922561b436c449de1a0bd1f5bd768248c # v0.76.1
 
       - name: Run Updatecli in Apply mode
         run: updatecli apply --config .github/updatecli-bump-golang.yml

.github/workflows/golangci-lint.yml

@@ -25,7 +25,7 @@ jobs:
           go-version-file: .go-version
 
       - name: golangci-lint
-        uses: golangci/golangci-lint-action@v3
+        uses: golangci/golangci-lint-action@v6
         with:
           # Optional: version of golangci-lint to use in form of v1.2 or v1.2.3 or `latest` to use the latest version
           version: v1.55.2
@@ -39,8 +39,5 @@ jobs:
           # into fixing all linting issues in the whole file instead.
           args: --timeout=30m --whole-files
 
-          # Optional: if set to true then the action will use pre-installed Go.
-          skip-go-installation: true
-
           # Optional: show only new issues if it's a pull request. The default value is `false`.
           only-new-issues: true

NOTICE.txt

@@ -0,0 +1,36 @@
+# Kind can be one of:
+# - breaking-change: a change to previously-documented behavior
+# - deprecation: functionality that is being removed in a later release
+# - bug-fix: fixes a problem in a previous version
+# - enhancement: extends functionality but does not break or fix existing behavior
+# - feature: new functionality
+# - known-issue: problems that we are aware of in a given version
+# - security: impacts on the security of a product or a user’s deployment.
+# - upgrade: important information for someone upgrading from a prior version
+# - other: does not fit into any of the other categories
+kind: enhancement
+
+# Change summary; a 80ish characters long description of the change.
+summary: Fleet Server component now uses policy output configuration to communicate with Elasticsearch
+
+# Long description; in case the summary is not enough to describe the change
+# this field accommodate a description without length limits.
+# NOTE: This field will be rendered only for breaking-change and known-issue kinds at the moment.
+description: |
+  Alter how elatic-agent passes the fleet-server output component so that the policy's output is used.
+  In cases where fleet-server encounters an error when trying to use the policy's output it will use
+  the configuration specified during enrollment as a fallback. In cases where it uses the fallback
+  the policy's output is periodically retested and used if it's successful.
+
+# Affected component; a word indicating the component this changeset affects.
+component:
+
+# PR URL; optional; the PR number that added the changeset.
+# If not present is automatically filled by the tooling finding the PR where this changelog fragment has been added.
+# NOTE: the tooling supports backports, so it's able to fill the original PR number instead of the backport PR number.
+# Please provide it if you are adding a fragment for a different PR.
+pr: https://github.com/elastic/elastic-agent/pull/4643
+
+# Issue URL; optional; the GitHub issue related to this changeset (either closes or is part of).
+# If not present is automatically filled by the tooling with the issue linked to the PR number.
+issue: https://github.com/elastic/elastic-agent/issue/2784

changelog/fragments/1714505172-Send-fleet-server-component-output-under-bootstrap-key.yaml

@@ -0,0 +1,32 @@
+# Kind can be one of:
+# - breaking-change: a change to previously-documented behavior
+# - deprecation: functionality that is being removed in a later release
+# - bug-fix: fixes a problem in a previous version
+# - enhancement: extends functionality but does not break or fix existing behavior
+# - feature: new functionality
+# - known-issue: problems that we are aware of in a given version
+# - security: impacts on the security of a product or a user’s deployment.
+# - upgrade: important information for someone upgrading from a prior version
+# - other: does not fit into any of the other categories
+kind: feature
+
+# Change summary; a 80ish characters long description of the change.
+summary: Add cloudbeat asset inventory aws
+
+# Long description; in case the summary is not enough to describe the change
+# this field accommodate a description without length limits.
+# NOTE: This field will be rendered only for breaking-change and known-issue kinds at the moment.
+#description:
+
+# Affected component; usually one of "elastic-agent", "fleet-server", "filebeat", "metricbeat", "auditbeat", "all", etc.
+component: cloudbeat
+
+# PR URL; optional; the PR number that added the changeset.
+# If not present is automatically filled by the tooling finding the PR where this changelog fragment has been added.
+# NOTE: the tooling supports backports, so it's able to fill the original PR number instead of the backport PR number.
+# Please provide it if you are adding a fragment for a different PR.
+#pr: https://github.com/owner/repo/1234
+
+# Issue URL; optional; the GitHub issue related to this changeset (either closes or is part of).
+# If not present is automatically filled by the tooling with the issue linked to the PR number.
+#issue: https://github.com/owner/repo/1234

changelog/fragments/1716543533-add-cloudbeat-asset-inventory-aws.yaml

@@ -88,3 +88,61 @@ its API key to use for communication. The new `fleet.yml` still includes the `fl
 but this time the `fleet.server.bootstrap: false` is set.
 . `enroll` command then either restarts the running Elatic Agent daemon if one was running
 from Step 2, or it stops the spawned `run` subprocess and returns.
+
+=== Elasticsearch output
+
+The options passed that are used to specify fleet-server initially connects to elasticsearch are:
+
+- `--fleet-server-es`
+- `--fleet-server-es-ca`
+- `--fleet-server-es-ca-trusted-fingerprint`
+- `--fleet-server-es-insecure`
+- `--fleet-server-es-cert`
+- `--fleet-server-es-cert-key`
+- `--fleet-server-es-service-token`
+- `--fleet-server-es-service-token-path`
+- `--proxy-url`
+- `--proxy-disabled`
+- `--proxy-header`
+
+These options are always passed under a `bootstrap` attribute in the output when elastic-agent is passing config to fleet-server.
+When the fleet-server recieves an output block, it will inject any keys that are missing from the top level output but are specified in the `bootstrap` block
+After injecting the keys from bootstrap, fleet-server will test connecting the Elasticsearch with the output.
+If the test fails, the values under the `bootstrap` attribute are used as the output and fleet-server will periodically retest the output in case the error was caused by a temporary network issue.
+Note that if `--fleet-server-es-insecure` is specified, and the output in the policy contains one or more CA, or a CA fingerprint, the `--fleet-server-es-insecure` flag is ignored.
+
+An example of this sequence is sequence is:
+
+1) elastic-agent starts fleet-server and sends an output block that looks similar to:
+```yaml
+output:
+  bootstrap:
+    service_token: VALUE
+    hosts: ["HOST"]
+```
+
+2) fleet-server injects attributes into the top level from bootstrap if they are missing, resulting in
+```yaml
+output:
+  service_token: VALUE
+  hosts: ["HOST"]
+```
+
+3) fleet-server connects to Elasticsearch with the output block
+4) elastic-agent enrolls and recieves its policy
+5) elastic-agent sends configuration generated from the policy to fleet-server, this may result in the output as follows:
+```yaml
+output:
+  hosts: ["HOST", "HOST2"]
+  bootstrap:
+    service_token: VALUE
+    hosts: ["HOST"]
+```
+
+6) fleet-server will inject missing values resulting in:
+```yaml
+output:
+  service_token: VALUE
+  hosts: ["HOST", "HOST2"]
+```
+ 7) fleet-server tests and uses the resulting output block.