Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add tutorial for standalone agent on serverless monitoring nginx #906

Merged
merged 5 commits into from
Feb 13, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -1,21 +1,22 @@
[[example-standalone-monitor-nginx]]
= Example: Use standalone {agent} to monitor nginx
= Example: Use standalone {agent} with {ess} to monitor nginx

This guide walks you through a simple monitoring scenario so you can learn the basics of setting up standalone {agent}, using it to work with {ecloud} and an Elastic integration.
This guide walks you through a simple monitoring scenario so you can learn the basics of setting up standalone {agent}, using it to work with {ess} and an Elastic integration.

Following these steps, you'll deploy the {stack}, install a standalone {agent} on a host to monitor an nginx web server instance, and access visualizations based on the collected logs.

. <<nginx-guide-install-nginx,Install nginx>>.
. <<nginx-guide-sign-up,Create an {ecloud} deployment>>.
. <<nginx-guide-create-policy,Create an {agent} policy>>.
. <<nginx-guide-add-integration,Add the Nginx Integration>>.
. <<nginx-guide-configure-standalone-agent,Configure standalone {agent}>>.
. <<nginx-guide-confirm-agent-data,Confirm that your {agent} data is flowing>>.
. <<nginx-guide-view-system-data,View your system data>>.
. <<nginx-guide-view-nginx-data,View your nginx logging data>>.
. <<nginx-guide-install-nginx-ess,Install nginx>>.
. <<nginx-guide-sign-up-ess,Create an {ecloud} deployment>>.
. <<nginx-guide-create-api-key-ess,Create an {ecloud} API key.>>
. <<nginx-guide-create-policy-ess,Create an {agent} policy>>.
. <<nginx-guide-add-integration-ess,Add the Nginx Integration>>.
. <<nginx-guide-configure-standalone-agent-ess,Configure standalone {agent}>>.
. <<nginx-guide-confirm-agent-data-ess,Confirm that your {agent} data is flowing>>.
. <<nginx-guide-view-system-data-ess,View your system data>>.
. <<nginx-guide-view-nginx-data-ess,View your nginx logging data>>.

[discrete]
[[nginx-guide-prereqs]]
[[nginx-guide-prereqs-ess]]
=== Prerequisites

To get started, you need:
Expand All @@ -24,7 +25,7 @@ To get started, you need:
. A Linux host machine on which you'll install an nginx web server. The commands in this guide use an Ubuntu image but any Linux distribution should be fine.

[discrete]
[[nginx-guide-install-nginx]]
[[nginx-guide-install-nginx-ess]]
=== Step 1: Install nginx

To start, we'll set up a basic link:https://docs.nginx.com/nginx/admin-guide/web-server/[nginx web server].
Expand All @@ -42,12 +43,12 @@ sudo apt install nginx
image::images/guide-nginx-welcome.png["Browser window showing Welcome to nginx!"]

[discrete]
[[nginx-guide-sign-up]]
[[nginx-guide-sign-up-ess]]
=== Step 2: Create an {ecloud} deployment

NOTE: If you've already signed up for a trial deployment you can skip this step.

Now that your web server is running, let's get set up to monitor it in {ecloud}. An {ecloud} deployment offers you all of the features of the {stack} as a hosted service. To test drive your first deployment, sign up for a free {ecloud} trial:
Now that your web server is running, let's get set up to monitor it in {ecloud}. An {ecloud} {ess} deployment offers you all of the features of the {stack} as a hosted service. To test drive your first deployment, sign up for a free {ecloud} trial:

. Go to our link:https://cloud.elastic.co/registration?elektra=guide-welcome-cta[{ecloud} Trial] page.

Expand All @@ -56,18 +57,33 @@ Now that your web server is running, let's get set up to monitor it in {ecloud}.
[role="screenshot"]
image::images/guide-sign-up-trial.png["Start your free Elastic Cloud trial",width="50%"]

. After you've link:https://cloud.elastic.co/login[logged in], you can create a deployment. Give your deployment a name and select *Create deployment*.
+
[role="screenshot"]
image::images/guide-create-first-deployment.png["Create your first deployment",width="70%"]
. After you've link:https://cloud.elastic.co/login[logged in], select *Create deployment* and give your deployment a name. You can leave the default options or select a different cloud provider, region, hardware profile, or version.

. Select *Create deployment*.

. While the deployment sets up, make a note of your `elastic` superuser password and keep it in a safe place.

. Once the deployment is ready, select *Continue*. At this point, you access {kib} and a selection of setup guides.

[discrete]
[[nginx-guide-create-policy]]
=== Step 3: Create an {agent} policy
[[nginx-guide-create-api-key-ess]]
=== Step 3: Create an {ecloud} API key

. From the {kib} menu and go to *Stack Management* -> *API keys*.

. Select *Create API key*.

. Give the key a name, for example `nginx example API key`.

. Leave the other default options and select *Create API key*.

. In the *Create API key* confirmation dialog, change the dropdown menu setting from `Encoded` to `Beats`. This sets the API key format for communication between {agent} (which is based on {beats}) and {es}.

. Copy the generated API key and store it in a safe place. You'll use it in a later step.

[discrete]
[[nginx-guide-create-policy-ess]]
=== Step 4: Create an {agent} policy

{agent} is a single, unified way to add monitoring for logs, metrics, and other types of data to a host. It can also protect hosts from security threats, query data from operating systems, and more. A single agent makes it easy and fast to deploy monitoring across your infrastructure. Each agent has a single policy (a collection of input settings) that you can update to add integrations for new data sources, security protections, and more.

Expand All @@ -82,15 +98,15 @@ image::images/guide-agent-policies.png["Agent policies tab in Fleet"]
image::images/guide-create-agent-policy.png["Create agent policy UI"]

[discrete]
[[nginx-guide-add-integration]]
=== Step 4: Add the Nginx Integration
[[nginx-guide-add-integration-ess]]
=== Step 5: Add the Nginx Integration

Elastic integrations are a streamlined way to connect your data from popular services and platforms to the {stack}, including nginx.

. From the **{fleet} -> Agent policies** tab, click the link for your new `nginx-policy`.
+
image::images/guide-nginx-policy.png["The nginx-policy UI with integrations tab selected"]
. Note that `system-` integration is included because you opted earlier to collect system logs and metrics.
. Note that the System integration (`system-1`) is included because you opted earlier to collect system logs and metrics.
. Click **Add integration**.
. On the Integrations page search for "nginx".
+
Expand All @@ -100,16 +116,16 @@ image::images/guide-integrations-page.png["Integrations page with nginx in the s
. Click the link to **Add integration only (skip agent installation)**. You'll install standalone {agent} in a later step.
. Here, you can select options such as the paths to where your nginx logs are stored, whether or not to collect metrics data, and various other settings.
+
For now, leave all of the default settings and click **Save and continue** to add the Nginx Integration to your `nginx-policy` policy.
For now, leave all of the default settings and click **Save and continue** to add the Nginx integration to your `nginx-policy` policy.
+
image::images/guide-add-nginx-integration.png["Add Nginx Integration UI"]
. In the confirmation dialog, select to **Add {agent} later**.
+
image::images/guide-nginx-integration-added.png["Nginx Integration added confirmation UI with Add {agent} later selected."]

[discrete]
[[nginx-guide-configure-standalone-agent]]
=== Step 5: Configure standalone {agent}
[[nginx-guide-configure-standalone-agent-ess]]
=== Step 6: Configure standalone {agent}

Rather than opt for {fleet} to centrally manage {agent}, you'll configure an agent to run in standalone mode, so it will be managed by hand.

Expand All @@ -120,7 +136,7 @@ Rather than opt for {fleet} to centrally manage {agent}, you'll configure an age
image::images/guide-add-agent-standalone01.png["Add agent UI with nginx-policy and Run-standalone selected."]
. For the **Configure the agent** step, choose **Download Policy**. Save the `elastic-agent.yml` file to a directory on the host where you'll install nginx for monitoring.
+
Notice that the policy contains all of the input, output, and other settings for the Nginx and System Integrations. If you already have a standalone agent installed on a host with an existing {agent} policy, you can use the method described here to add a new integration. Just add the settings from the **Configure the agent** step to your existing `elastic-agent.yml` file.
Have a look inside the policy file and notice that it contains all of the input, output, and other settings for the Nginx and System integrations. If you already have a standalone agent installed on a host with an existing {agent} policy, you can use the method described here to add a new integration. Just add the settings from the **Configure the agent** step to your existing `elastic-agent.yml` file.
. For the **Install {agent} on your host** step, select the tab for your host operating system and run the commands on your host.
+
image::images/guide-install-agent-on-host.png["Install {agent} on your host step, showing tabs with the commands for different operating systems."]
Expand Down Expand Up @@ -148,8 +164,23 @@ elastic-agent status
+
Since you're running the agent in standalone mode the `Not enrolled into Fleet` message is expected.
. Open the `elastic-agent.yml` policy file that you saved.
. Near the top of the file, replace `${ES_USERNAME}` with the username for the {ecloud} deployment that you created in <<nginx-guide-sign-up,Step 2>>. The default is `elastic`.
. Similarly, replace `${ES_PASSWORD}` with your superuser password.
. Near the top of the file, replace:
+
[source,yaml]
----
username: '${ES_USERNAME}'
password: '${ES_PASSWORD}'
----
+
with:
+
[source,yaml]
----
api_key: '<your-api-key>'
----
+
where `your-api-key` is the API key that you generated in <<nginx-guide-create-api-key-ess>>.

. Find the location of the default `elastic-agent.yml` policy file that is included in your {agent} install. Install directories for each platform are described in <<installation-layout,Installation layout>>. In our example Ubuntu image the default policy file can be found in `/etc/elastic-agent/elastic-agent.yml`.
. Replace the default policy file with the version that you downloaded and updated. For example:
+
Expand Down Expand Up @@ -187,8 +218,8 @@ message: Running
----

[discrete]
[[nginx-guide-confirm-agent-data]]
=== Step 6: Confirm that your {agent} data is flowing
[[nginx-guide-confirm-agent-data-ess]]
=== Step 7: Confirm that your {agent} data is flowing

Now that {agent} is running, it's time to confirm that the agent data is flowing into {es}.

Expand All @@ -208,26 +239,28 @@ like the agent logs, the agent metrics should be flowing into {es} and visible i
image::images/guide-agent-metrics-flowing.png["Kibana Dashboard shows agent metrics are flowing into Elasticsearch."]

[discrete]
[[nginx-guide-view-system-data]]
=== Step 7: View your system data
[[nginx-guide-view-system-data-ess]]
=== Step 8: View your system data

In the step to <<nginx-guide-create-policy,create an {agent} policy>> you chose to collect system logs and metrics, so you can access those now.
In the step to <<nginx-guide-create-policy-ess,create an {agent} policy>> you chose to collect system logs and metrics, so you can access those now.

. View your system logs.
.. Open the {kib} menu and go to **Management -> Integrations -> Installed integrations**.
.. Select the **System** card and open the **Assets** tab. This is a quick way to access all of the dashboards, saved searches, and visualizations that come with each integration.
.. Select `[Logs System] Syslog dashboard`. The {kib} Dashboard opens with visualizations of Syslog events, hostnames and processes, and more.
.. Select `[Logs System] Syslog dashboard`.
.. Select the calandar icon and change the time setting to `Today`. The {kib} Dashboard shows visualizations of Syslog events, hostnames and processes, and more.
. View your system metrics.

.. Return to **Management -> Integrations -> Installed integrations**.
.. Select the **System** card and open the **Assets** tab.
.. Select `[Metrics System] Host overview`. The {kib} Dashboard opens with visualizations of host metrics including CPU usage, memory usage, running processes, and others.
.. This time, select `[Metrics System] Host overview`.
.. Select the calandar icon and change the time setting to `Today`. The {kib} Dashboard shows visualizations of host metrics including CPU usage, memory usage, running processes, and others.
+
image::images/guide-system-metrics-dashboard.png["The System metrics host overview showing CPU usage, memory usage, and other visualizations"]

[discrete]
[[nginx-guide-view-nginx-data]]
=== Step 8: View your nginx logging data
[[nginx-guide-view-nginx-data-ess]]
=== Step 9: View your nginx logging data

Now let's view your nginx logging data.

Expand All @@ -240,7 +273,7 @@ image::images/guide-nginx-logs-dashboard.png["The nginx logs dashboard shows var
+
image::images/guide-nginx-browser-breakdown.png["Kibana Dashboard shows agent metrics are flowing into Elasticsearch."]

You have now successfully set up monitoring for nginx logs using standalone {agent} and an {ecloud} deployment.
Congratulations! You have successfully set up monitoring for nginx using standalone {agent} and an {ecloud} deployment.

[discrete]
=== What's next?
Expand Down
Loading