Example: Install standalone {agent} on Kubernetes using Helm

preview::[]

This example demonstrates how to install standalone {agent} on a Kubernetes system using a Helm chart, gather Kubernetes metrics and send them to an {es} cluster in {ecloud}, and then view visualizations of those metrics in {kib}.

For an overview of the {agent} Helm chart and its benefits, refer to [install-on-kubernetes-using-helm].

This guide takes you through these steps:

Install {agent}
Upgrade your {agent} configuration
Change {agent}'s running mode
Tidy up

Prerequisites

To get started, you need:

A local install of the Helm {k8s} package manager.
An {ecloud} hosted {es} cluster on version 8.16 or higher.
An {es} API key.
An active {k8s} cluster.
A local clone of the elastic/elastic-agent GitHub repository. Make sure to use the 8.16 branch to ensure that {agent} has full compatibility with the Helm chart.

Install {agent}

Open your {ecloud} deployment, and from the navigation menu select Manage this deployment.
In the Applications section, copy the {es} endpoint and make a note of the endpoint value.
Open a terminal shell and change into a directory in your local clone of the elastic-agent repo.
Copy this command.
```
helm install demo ./deploy/helm/elastic-agent \
--set kubernetes.enabled=true \
--set outputs.default.type=ESPlainAuthAPI \
--set outputs.default.url=<ES-endpoint>:443 \
--set outputs.default.api_key="API_KEY"
```
Note that the command has these properties:
- helm install runs the Helm CLI install tool.
- demo gives a name to the installed chart. You can choose any name.
- ./deploy/helm/elastic-agent is a local path to the Helm chart to install (in time it’s planned to have a public URL for the chart).
- --set kubernetes.enabled=true enables the {k8s} integration. The CLI parameter overrides the default false value for kubernetes.enabled in the {agent} values.yaml file.
- --set outputs.default.type=ESPlainAuthAPI sets the authentication method for the {es} output to require an API key (again, overriding the value set by default in the {agent} values.yaml file).
- --set outputs.default.url=<ES-endpoint>:443 sets the address of your {ecloud} deployment, where {agent} will send its output over port 443.
- --set outputs.default.api_key="API_KEY" sets the API key that {agent} will use to authenticate with your {es} cluster.
  
  Tip
  For a full list of all available YAML settings and descriptions, refer to the {agent} Helm Chart Readme.

Update the command to replace:

<ES-endpoint> with the {es} endpoint value that you copied earlier.

<API_KEY> with your API key name.

After your updates, the command should look something like this:

helm install demo ./deploy/helm/elastic-agent \
--set kubernetes.enabled=true \
--set outputs.default.type=ESPlainAuthAPI \
--set outputs.default.url=https://demo.es.us-central1.gcp.foundit.no:443 \
--set outputs.default.api_key="A6ecaHNTJUFFcJI6esf4:5HJPxxxxxxxPS4KwSBeVEs"

Run the command.

The command output should confirm that three {agents} have been installed as well as the {k8s} integration:

...
Installed agents:
  - clusterWide [deployment - standalone mode]
  - ksmSharded [statefulset - standalone mode]
  - perNode [daemonset - standalone mode]

Installed integrations:
  - kubernetes [built-in chart integration]
...

Run the kubectl get pods -n default command to confirm that the {agent} pods are running:

NAME                                      READY   STATUS    RESTARTS   AGE
agent-clusterwide-demo-77c65f6c7b-trdms   1/1     Running   0          5m18s
agent-ksmsharded-demo-0                   2/2     Running   0          5m18s
agent-pernode-demo-c7d75                  1/1     Running   0          5m18s

In your {ecloud} deployment, from the {kib} menu open the Integrations page.
Run a search for Kubernetes and then select the {k8s} integration card.
On the {k8s} integration page, select Install Kubernetes assets. This installs the dashboards, {es} indexes, and other assets used to monitor your {k8s} cluster.
On the {k8s} integration page, open the Assets tab and select the [Metrics Kubernetes] Nodes dashboard.

On the dashboard, you can view the status of your {k8s} nodes, including metrics on memory, CPU, and filesystem usage, network throughput, and more.
On the {k8s} integration page, open the Assets tab and select the [Metrics Kubernetes] Pods dashboard. As with the nodes dashboard, on this dashboard you can view the status of your {k8s} pods, including various metrics on memory, CPU, and network throughput.

Upgrade your {agent} configuration

Now that you have {agent} installed, collecting, and sending data successfully, let’s try changing the agent configuration settings.

In the previous install example, three {agent} nodes were installed. One of these nodes, agent-ksmsharded-demo-0, is installed to enable the kube-state-metrics service. Let’s suppose that you don’t need those metrics and would like to upgrade your configuration accordingly.

Copy the command that you used earlier to install {agent}:

helm install demo ./deploy/helm/elastic-agent \
--set kubernetes.enabled=true \
--set outputs.default.type=ESPlainAuthAPI \
--set outputs.default.url=<ES-endpoint>:443 \
--set outputs.default.api_key="API_KEY"

Update the command as follows:

Change install to upgrade.

Add a parameter --set kubernetes.state.enabled=false. This will override the default true value for the setting kubernetes.state in the {agent} values.yaml file.

helm upgrade demo ./deploy/helm/elastic-agent \
--set kubernetes.enabled=true \
--set kubernetes.state.enabled=false \
--set outputs.default.type=ESPlainAuthAPI \
--set outputs.default.url=<ES-endpoint>:443 \
--set outputs.default.api_key="API_KEY"

Run the command.

The command output should confirm that now only two {agents} are installed together with the {k8s} integration:

...
Installed agents:
  - clusterWide [deployment - standalone mode]
  - perNode [daemonset - standalone mode]

Installed integrations:
  - kubernetes [built-in chart integration]
...

You’ve upgraded your configuration to run only two {agents}, without the kube-state-metrics service. You can similarly upgrade your agent to change other settings defined in the in the {agent} values.yaml file.

Change {agent}'s running mode

By default {agent} runs under the elastic user account. For some use cases you may want to temporarily change an agent to run with higher privileges.

Run the kubectl get pods -n default command to view the running {agent} pods:

NAME                                      READY   STATUS    RESTARTS   AGE
agent-clusterwide-demo-77c65f6c7b-trdms   1/1     Running   0          5m18s
agent-pernode-demo-c7d75                  1/1     Running   0          5m18s

Now, run the kubectl exec command to enter one of the running {agents}, substituting the correct pod name returned from the previous command. For example:
```
kubectl exec -it pods/agent-pernode-demo-c7d75 -- bash
```

From inside the pod, run the Linux ps aux command to view the running processes.

ps aux

The results should be similar to the following:

USER         PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
elastic+           1  0.0  0.0   1936   416 ?        Ss   21:04   0:00 /usr/bin/tini -- /usr/local/bin/docker-entrypoint -c /etc/elastic-agent/agent.yml -e
elastic+          10  0.2  1.3 2555252 132804 ?      Sl   21:04   0:13 elastic-agent container -c /etc/elastic-agent/agent.yml -e
elastic+          37  0.6  2.0 2330112 208468 ?      Sl   21:04   0:37 /usr/share/elastic-agent/data/elastic-agent-d99b09/components/agentbeat metricbeat -E
elastic+          38  0.2  1.7 2190072 177780 ?      Sl   21:04   0:13 /usr/share/elastic-agent/data/elastic-agent-d99b09/components/agentbeat filebeat -E se
elastic+          56  0.1  1.7 2190136 175896 ?      Sl   21:04   0:11 /usr/share/elastic-agent/data/elastic-agent-d99b09/components/agentbeat metricbeat -E
elastic+          68  0.1  1.8 2190392 184140 ?      Sl   21:04   0:12 /usr/share/elastic-agent/data/elastic-agent-d99b09/components/agentbeat metricbeat -E
elastic+          78  0.7  2.0 2330496 204964 ?      Sl   21:04   0:48 /usr/share/elastic-agent/data/elastic-agent-d99b09/components/agentbeat filebeat -E se
elastic+         535  0.0  0.0   3884  3012 pts/0    Ss   22:47   0:00 bash
elastic+         543  0.0  0.0   5480  2360 pts/0    R+   22:47   0:00 ps aux

In the command output, note that {agent} is currently running as the elastic user:

elastic+          10  0.2  1.3 2555252 132804 ?      Sl   21:04   0:13 elastic-agent container -c /etc/elastic-agent/agent.yml -e

Run exit to leave the {agent} pod.

Run the helm upgrade command again, this time adding the parameter --set agent.unprivileged=false to override the default true value for that setting.

helm upgrade demo ./deploy/helm/elastic-agent \
--set kubernetes.enabled=true \
--set kubernetes.state.enabled=false \
--set outputs.default.type=ESPlainAuthAPI \
--set outputs.default.url=<ES-endpoint>:443 \
--set outputs.default.api_key="API_KEY" \
--set agent.unprivileged=false

Run the kubectl get pods -n default command to view the running {agent} pods:

NAME                                      READY   STATUS    RESTARTS   AGE
agent-clusterwide-demo-77c65f6c7b-trdms   1/1     Running   0          5m18s
agent-pernode-demo-s6s7z                  1/1     Running   0          5m18s

Re-run the kubectl exec command to enter one of the running {agents}, substituting the correct pod name. For example:
```
kubectl exec -it pods/agent-pernode-demo-s6s7z -- bash
```

From inside the pod, run the Linux ps aux command to view the running processes.

USER         PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root       1  0.0  0.0   1936   452 ?        Ss   23:10   0:00 /usr/bin/tini -- /usr/local/bin/docker-entrypoint -c /etc/elastic-agent/agent.yml -e
root       9  0.9  1.3 2488368 135920 ?      Sl   23:10   0:01 elastic-agent container -c /etc/elastic-agent/agent.yml -e
root      27  0.9  1.9 2255804 203128 ?      Sl   23:10   0:01 /usr/share/elastic-agent/data/elastic-agent-d99b09/components/agentbeat metricbeat -E
root      44  0.3  1.8 2116148 187432 ?      Sl   23:10   0:00 /usr/share/elastic-agent/data/elastic-agent-d99b09/components/agentbeat metricbeat -E
root      64  0.3  1.8 2263868 188892 ?      Sl   23:10   0:00 /usr/share/elastic-agent/data/elastic-agent-d99b09/components/agentbeat metricbeat -E
root      76  0.4  1.8 2190136 190972 ?      Sl   23:10   0:00 /usr/share/elastic-agent/data/elastic-agent-d99b09/components/agentbeat filebeat -E se
root     100  1.2  2.0 2256316 207692 ?      Sl   23:10   0:01 /usr/share/elastic-agent/data/elastic-agent-d99b09/components/agentbeat filebeat -E se
root     142  0.0  0.0   3752  3068 pts/0    Ss   23:12   0:00 bash
root     149  0.0  0.0   5480  2376 pts/0    R+   23:13   0:00 ps aux

Run exit to leave the {agent} pod.

You’ve upgraded the {agent} privileges to run as root. To change the settings back, you can re-run the helm upgrade command with --set agent.unprivileged=true to return to the default unprivileged mode.

Tidy up

After you’ve run through this example, run the helm uninstall command to uninstall {agent}.

helm uninstall demo

The uninstall should be confirmed as shown:

release "demo" uninstalled

As a reminder, for full details about using the {agent} Helm chart refer to the {agent} Helm Chart Readme.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

example-kubernetes-standalone-agent-helm.asciidoc

example-kubernetes-standalone-agent-helm.asciidoc

Example: Install standalone {agent} on Kubernetes using Helm

Prerequisites

Install {agent}

Upgrade your {agent} configuration

Change {agent}'s running mode

Tidy up

Files

example-kubernetes-standalone-agent-helm.asciidoc

Latest commit

History

example-kubernetes-standalone-agent-helm.asciidoc

File metadata and controls

Example: Install standalone {agent} on Kubernetes using Helm

Prerequisites

Install {agent}

Upgrade your {agent} configuration

Change {agent}'s running mode

Tidy up