[Fix] Agent incapable of running on Azure Container Instances (#3576)

What this change is introducing on top of bringing back work introduced in #3084 is change of ordrer for some operations.

Changing owner of a file, discards capabilities set. This becomes a problem with heartbeat as it needs setuid and netraw capabilities to perform properly.
So setting capabilities was moved after chown.

Author: michalpristas

changelog/fragments/1689328899-Elastic-Agent-container-runs-on-Azure-Container-Instances-.yaml

@@ -0,0 +1,31 @@
+# Kind can be one of:
+# - breaking-change: a change to previously-documented behavior
+# - deprecation: functionality that is being removed in a later release
+# - bug-fix: fixes a problem in a previous version
+# - enhancement: extends functionality but does not break or fix existing behavior
+# - feature: new functionality
+# - known-issue: problems that we are aware of in a given version
+# - security: impacts on the security of a product or a user’s deployment.
+# - upgrade: important information for someone upgrading from a prior version
+# - other: does not fit into any of the other categories
+kind: bug
+
+# Change summary; a 80ish characters long description of the change.
+summary: Elastic-Agent container runs on Azure Container Instances 
+
+# Long description; in case the summary is not enough to describe the change
+# this field accommodate a description without length limits.
+#description: 
+
+# Affected component; a word indicating the component this changeset affects.
+component: elastic-agent
+
+# PR number; optional; the PR number that added the changeset.
+# If not present is automatically filled by the tooling finding the PR where this changelog fragment has been added.
+# NOTE: the tooling supports backports, so it's able to fill the original PR number instead of the backport PR number.
+# Please provide it if you are adding a fragment for a different PR.
+pr: 3576
+
+# Issue number; optional; the GitHub issue related to this changeset (either closes or is part of).
+# If not present is automatically filled by the tooling with the issue linked to the PR number.
+issue: 82

dev-tools/packaging/templates/docker/Dockerfile.elastic-agent.tmpl

@@ -9,7 +9,6 @@ FROM {{ .buildFrom }} AS home
 COPY beat {{ $beatHome }}
 
 RUN mkdir -p {{ $beatHome }}/data {{ $beatHome }}/data/elastic-agent-{{ commit_short }}/logs && \
-    chown -R root:root {{ $beatHome }} && \
     find {{ $beatHome }} -type d -exec chmod 0755 {} \; && \
     find {{ $beatHome }} -type f -exec chmod 0644 {} \; && \
     find {{ $beatHome }}/data -type d -exec chmod 0770 {} \; && \
@@ -127,25 +126,16 @@ COPY --from=home {{ $beatHome }}/NOTICE.txt /licenses
 COPY --from=home /opt /opt
 {{- end }}
 
-
-RUN setcap cap_net_raw,cap_setuid+p {{ $beatHome }}/data/elastic-agent-{{ commit_short }}/components/heartbeat && \
-{{- if .linux_capabilities }}
-# Since the beat is stored at the other end of a symlink we must follow the symlink first
-# For security reasons setcap does not support symlinks. This is smart in the general case
-# but in our specific case since we're building a trusted image from trusted binaries this is
-# fine. Thus, we use readlink to follow the link and setcap on the actual binary
-    readlink -f {{ $beatBinary }} | xargs setcap {{ .linux_capabilities }} && \
-{{- end }}
-true
-
 {{- if eq .user "root" }}
 {{- if contains .image_name "-cloud" }}
 # Generate folder for a stub command that will be overwritten at runtime
 RUN mkdir /app
 {{- end }}
 {{- else }}
-RUN groupadd --gid 1000 {{ .BeatName }}
-RUN useradd -M --uid 1000 --gid 1000 --groups 0 --home {{ $beatHome }} {{ .user }}
+RUN groupadd --gid 1000 {{ .BeatName }} && \
+    useradd -M --uid 1000 --gid 1000 --groups 0 --home {{ $beatHome }} {{ .user }} && \
+    chown -R {{ .user }}:{{ .user }} {{ $beatHome }} && \
+    true
 
 {{- if contains .image_name "-cloud" }}
 # Generate folder for a stub command that will be overwritten at runtime
@@ -154,6 +144,17 @@ RUN chown {{ .user }} /app
 {{- end }}
 {{- end }}
 
+# Keep this after any chown command, chown resets any applied capabilities
+RUN setcap cap_net_raw,cap_setuid+p {{ $beatHome }}/data/elastic-agent-{{ commit_short }}/components/heartbeat && \
+{{- if .linux_capabilities }}
+# Since the beat is stored at the other end of a symlink we must follow the symlink first
+# For security reasons setcap does not support symlinks. This is smart in the general case
+# but in our specific case since we're building a trusted image from trusted binaries this is
+# fine. Thus, we use readlink to follow the link and setcap on the actual binary
+   setcap {{ .linux_capabilities }}  $(readlink -f {{ $beatBinary }}) && \
+{{- end }}
+true
+
 {{- if (and (contains .image_name "-complete") (not (contains .from "ubi-minimal")))  }}
 USER root
 ENV NODE_PATH={{ $beatHome }}/.node