From 0a4cf9a786f8a469a3cfbccdb7cef9614c88a5ab Mon Sep 17 00:00:00 2001 From: jackshirazi Date: Thu, 13 Mar 2025 12:27:37 +0000 Subject: [PATCH 1/4] Update troubleshooting.md Initial cut --- docs/_edot-sdks/java/troubleshooting.md | 55 ++++++++++++++++++++++++- 1 file changed, 54 insertions(+), 1 deletion(-) diff --git a/docs/_edot-sdks/java/troubleshooting.md b/docs/_edot-sdks/java/troubleshooting.md index 9bd557ee..9db8a95f 100644 --- a/docs/_edot-sdks/java/troubleshooting.md +++ b/docs/_edot-sdks/java/troubleshooting.md @@ -7,4 +7,57 @@ parent: EDOT Java # Troubleshooting the EDOT Java Agent -TODO \ No newline at end of file +The sections below are in the order you should follow, unless you have already identified the section you need. + +This guide assumes you have tested the other components in the route from application+agent to Elastic Observability (eg collector or APM server, Elasticsearch, and Kibana) and that the problem has been isolated to the application+agent. + +## General + +Ensure you have set a service name (eg `-Dotel.service.name=Service1` or environment variable `OTEL_SERVICE_NAME` set to `Service1`) otherwise by default the data (traces, metrics, logs) will be sent to `unknown_service_java` - you may be getting data but it may all be under that service + +## Connectivity + +### To endpoint + +1. Check _from_ the host/VM/pod/container/image running the app, that connectivity is available to the APM server or collector. The examples here use a default URL, which you should replace with the endpoint you are using: + +- OpenTelemetry or EDOT collector: `curl -i http://127.0.0.1:4318/v1/traces -X POST -d '{}' -H content-type:application/json` +- APM server on http: `curl --verbose -X GET http://127.0.0.1:8200` +- APM server on https with secret token: `curl -X POST http://127.0.0.1:8200/ -H "Authorization: Bearer "` +- APM server on https with API key: `curl -X POST http://127.0.0.1:8200/ -H "Authorization: ApiKey "` + +The collector should produce output similar to +``` +{"partialSuccess":{}} +``` + +The APM server should produce output similar to +``` +{ + "build_date": "2021-12-18T19:59:06Z", + "build_sha": "24fe620eeff5a19e2133c940c7e5ce1ceddb1445", + "publish_ready": true, + "version": "8.17.3" +} +``` + +### Authentication + +In all cases, use the "APM server on https" endpoint connectivity test described above to confirm you are using the correct credentials + +## Is it the agent? + +Determine if the issue is related to the agent by + +1. Starting the application with no agent and seeing if the issue is not present, but then the issue is again present when restarting with the agent +2. Check end-to-end connectivity without the agent by running one or more of the example apps in https://github.com/elastic/elastic-otel-java/blob/main/examples/troubleshooting/README.md . These use the OpenTelemetry SDK rather than the auto-instrumentation, ie there is no agent present, and create traces, metrics and logs, so provide confirmation that the issue is specific to the agent or can otherwise identify that the issue is something else + +## Agent DEBUG + +Debug output is enabled with `-Dotel.javaagent.debug=true` or environment variable `OTEL_JAVAAGENT_DEBUG` to `true`. + +Once debug is enabled, look for: +- Errors and exceptions +- For the expected traces or metrics - or lack of them (maybe the [technology isn't instrumented?](https://github.com/open-telemetry/opentelemetry-java-instrumentation/blob/main/docs/supported-libraries.md)) + + From 850a8b8e248d4edbae95a2e3d43d3175001d6fc2 Mon Sep 17 00:00:00 2001 From: jackshirazi Date: Thu, 13 Mar 2025 14:33:23 +0000 Subject: [PATCH 2/4] Update docs/_edot-sdks/java/troubleshooting.md Co-authored-by: SylvainJuge <763082+SylvainJuge@users.noreply.github.com> --- docs/_edot-sdks/java/troubleshooting.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/_edot-sdks/java/troubleshooting.md b/docs/_edot-sdks/java/troubleshooting.md index 9db8a95f..0b1cf02a 100644 --- a/docs/_edot-sdks/java/troubleshooting.md +++ b/docs/_edot-sdks/java/troubleshooting.md @@ -21,10 +21,10 @@ Ensure you have set a service name (eg `-Dotel.service.name=Service1` or environ 1. Check _from_ the host/VM/pod/container/image running the app, that connectivity is available to the APM server or collector. The examples here use a default URL, which you should replace with the endpoint you are using: -- OpenTelemetry or EDOT collector: `curl -i http://127.0.0.1:4318/v1/traces -X POST -d '{}' -H content-type:application/json` -- APM server on http: `curl --verbose -X GET http://127.0.0.1:8200` -- APM server on https with secret token: `curl -X POST http://127.0.0.1:8200/ -H "Authorization: Bearer "` -- APM server on https with API key: `curl -X POST http://127.0.0.1:8200/ -H "Authorization: ApiKey "` +- OpenTelemetry or EDOT collector without authentication: `curl -i http://127.0.0.1:4318/v1/traces -X POST -d '{}' -H content-type:application/json` +- APM server without authentication: `curl --verbose -X GET http://127.0.0.1:8200` +- APM server with secret token authentication: `curl -X POST http://127.0.0.1:8200/ -H "Authorization: Bearer "` +- APM server with API key authentication: `curl -X POST http://127.0.0.1:8200/ -H "Authorization: ApiKey "` The collector should produce output similar to ``` From 2a33a2d5872c61e2378a2ba386773df46f7304c8 Mon Sep 17 00:00:00 2001 From: jackshirazi Date: Thu, 13 Mar 2025 14:34:15 +0000 Subject: [PATCH 3/4] Update docs/_edot-sdks/java/troubleshooting.md Co-authored-by: SylvainJuge <763082+SylvainJuge@users.noreply.github.com> --- docs/_edot-sdks/java/troubleshooting.md | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/docs/_edot-sdks/java/troubleshooting.md b/docs/_edot-sdks/java/troubleshooting.md index 0b1cf02a..2734e34b 100644 --- a/docs/_edot-sdks/java/troubleshooting.md +++ b/docs/_edot-sdks/java/troubleshooting.md @@ -15,11 +15,9 @@ This guide assumes you have tested the other components in the route from applic Ensure you have set a service name (eg `-Dotel.service.name=Service1` or environment variable `OTEL_SERVICE_NAME` set to `Service1`) otherwise by default the data (traces, metrics, logs) will be sent to `unknown_service_java` - you may be getting data but it may all be under that service -## Connectivity +## Connectivity to endpoint -### To endpoint - -1. Check _from_ the host/VM/pod/container/image running the app, that connectivity is available to the APM server or collector. The examples here use a default URL, which you should replace with the endpoint you are using: +Check _from_ the host/VM/pod/container/image running the app, that connectivity is available to the APM server or collector. The examples here use a default URL, which you should replace with the endpoint you are using: - OpenTelemetry or EDOT collector without authentication: `curl -i http://127.0.0.1:4318/v1/traces -X POST -d '{}' -H content-type:application/json` - APM server without authentication: `curl --verbose -X GET http://127.0.0.1:8200` From 18be2cd23c26d7dbcda888d007621c940cfb6800 Mon Sep 17 00:00:00 2001 From: jackshirazi Date: Thu, 13 Mar 2025 14:34:24 +0000 Subject: [PATCH 4/4] Update docs/_edot-sdks/java/troubleshooting.md Co-authored-by: SylvainJuge <763082+SylvainJuge@users.noreply.github.com> --- docs/_edot-sdks/java/troubleshooting.md | 3 --- 1 file changed, 3 deletions(-) diff --git a/docs/_edot-sdks/java/troubleshooting.md b/docs/_edot-sdks/java/troubleshooting.md index 2734e34b..81517311 100644 --- a/docs/_edot-sdks/java/troubleshooting.md +++ b/docs/_edot-sdks/java/troubleshooting.md @@ -39,9 +39,6 @@ The APM server should produce output similar to } ``` -### Authentication - -In all cases, use the "APM server on https" endpoint connectivity test described above to confirm you are using the correct credentials ## Is it the agent?