Merge pull request #107 from SylvainJuge/doc-java-performance

doc add java instrumentation overhead

Author: SylvainJuge

docs/_edot-sdks/java/migration.md

@@ -28,12 +28,13 @@ This documentation describes how to update applications that are currently using
 
 ## Migration steps
 
+- review all pros/cons of this migration guide including the [differences in performance overhead](./overhead).
 - (optional) migrate usages of Elastic APM Agent API with OpenTelemetry API in the application source code.
 - remove the `-javaagent:` argument containing [Elastic APM Java agent](https://www.elastic.co/guide/en/apm/agent/java/current/index.html) from the JVM arguments
 - replace configuration options using [reference](#option-reference) below, see [configuration](./configuration) for ways to provide those.
 - add `-javaagent:` argument to the JVM arguments to use EDOT Java and restart the application or follow [Kubernetes instructions](./setup/k8s) if applicable.
 - there is currently no EDOT equivalent for starting the agent with the [remote attach](https://www.elastic.co/guide/en/apm/agent/java/current/setup-attach-cli.html) capability. The `-javaagent:` option is the preferred startup mechanism.
-- there is a migration path for starting the agent with [self attach](https://www.elastic.co/guide/en/apm/agent/java/current/setup-attach-api.html), which is to use [runtime attachment](https://github.com/open-telemetry/opentelemetry-java-contrib/blob/main/runtime-attach/README.md). 
+- there is a migration path for starting the agent with [self attach](https://www.elastic.co/guide/en/apm/agent/java/current/setup-attach-api.html), which is to use [runtime attachment](https://github.com/open-telemetry/opentelemetry-java-contrib/blob/main/runtime-attach/README.md).
 
 ## Option reference
 

docs/_edot-sdks/java/overhead.md

@@ -0,0 +1,52 @@
+---
+title: Performance overhead
+layout: default
+nav_order: 8
+parent: EDOT Java
+---
+
+# Performance overhead
+
+While it is designed to and usually have minimal performance overhead, the EDOT Java agent, like any instrumentation agent,
+executes within the application process and thus has a small influence on the application performance. 
+
+This extra performance overhead depends on the application's technical architecture, its configuration and environment and 
+the load. Those factors are not something that are easy to reproduce on their own, and all applications are different so 
+it is impossible to provide a simple answer like "+1% extra CPU time and +100Mb of memory".
+
+The performance overhead on the JVM can be measured by the following high-level metrics:
+
+- application startup time
+- application response time, which can be directly perceived by users
+- CPU usage
+- Garbage collector activity: more memory allocation means more GC activity, thus increasing overall CPU usage and potentially reducing application responsiveness
+- Memory usage: how much more memory (heap/non-heap) is needed
+
+While we can't provide generically applicable, accurate numbers about the above, we execute synthetic benchmarks with a sample application
+which allows to provide an estimate, comparison between agents and an order of magnitude of the effective overhead.
+
+Those numbers are only provided as indicators, and you should not attempt to extrapolate them. You should however use
+them as a framework to evaluate and measure the overhead on your applications.
+
+For example, the application startup overhead going from 5s to 6s (+1s, +20%) does not mean an application having a startup time of
+15s will now start in 18s but that you can expect to have about at least one extra second of startup time and the overall
+impact remains limited.
+
+The following table compares the *classic Elastic APM Java Agent* with the `EDOT Java Agent` and the same benchmark without an agent.
+
+|                              | no agent  | EDOT Java instrumentation | Elastic APM Java agent |
+|------------------------------|-----------|---------------------------|------------------------|
+| startup time                 | 5.55 s    | 6.82 s                    | 6.87 s                 |
+| request latency (p95)        | 1.96 ms   | 2.06 ms                   | 2.08 ms                |
+| total system cpu utilization | 53.82 %   | 54.25 %                   | 56.92 %                |
+| total allocated memory       | 21.54 gb  | 26.37 gb                  | 22.15 gb               |
+| total GC pauses              | 106 ms    | 123 ms                    | 120 ms                 |
+| max heap used                | 436.71 mb | 478.46 mb                 | 573.92 mb              |
+
+The main difference between the two agents is that unlike EDOT, Elastic APM Java agent recycles in-memory data
+structures which allows to reduce the overall allocated memory and thus reduces a bit the overhead on the garbage collector.
+
+This difference is also the reason why we observe a difference in the maximum heap usage as more data structures are kept
+in-memory when possible and not recycled by the garbage collector. This however does not mean that Elastic APM Java agent _requires_
+about 100mb more of memory compared to EDOT, but that when there is no limitation on heap memory usage the agent will
+use available memory to minimize memory allocation.

docs/_edot-sdks/java/release.md

@@ -1,7 +1,7 @@
 ---
 title: Release Notes
 layout: default
-nav_order: 8
+nav_order: 9
 parent: EDOT Java
 ---