Skip to content

Commit 1410d2a

Browse files
michalpristaspchilaycombinator
authored
Embedded Otel Collector into Agent (#3800)
* basic otel with testdata * updated UT * updated UT * do not pass modifiers * lint: use Fprint instead of println" * path fixed * test cases * forgotten files * Added file ingestion test * Added file ingestion test * e2e with apm * imports * add group * add --config option to align with collector usage * Update testing/integration/otel_test.go Co-authored-by: Paolo Chilà <paolo.chila@elastic.co> * Update testing/integration/otel_test.go Co-authored-by: Paolo Chilà <paolo.chila@elastic.co> * Update pkg/testing/tools/estools/elasticsearch.go Co-authored-by: Shaunak Kashyap <ycombinator@gmail.com> * review * typos * fixed unit tests * try guarding against lower version * try guarding against lower version * try with windows * comment for skip * remove windows * quick review resolutions * check suffix for otel configs only * require yml or yaml config * e2e test with logger watching * some test cleanup * more cleanup * typo * name based resolution * e2e fixed * unit test fixed * missing files --------- Co-authored-by: Paolo Chilà <paolo.chila@elastic.co> Co-authored-by: Shaunak Kashyap <ycombinator@gmail.com>
1 parent ad7e1b5 commit 1410d2a

File tree

20 files changed

+24192
-4102
lines changed

20 files changed

+24192
-4102
lines changed

‎NOTICE.txt

+22,804-4,017
Large diffs are not rendered by default.

‎dev-tools/packaging/packages.yml

+16
Original file line numberDiff line numberDiff line change
@@ -42,6 +42,10 @@ shared:
4242
source: 'elastic-agent.yml'
4343
mode: 0600
4444
config: true
45+
/etc/{{.BeatName}}/otel.yml:
46+
source: 'otel.yml'
47+
mode: 0600
48+
config: true
4549
/etc/{{.BeatName}}/.elastic-agent.active.commit:
4650
content: >
4751
{{ commit }}
@@ -109,6 +113,10 @@ shared:
109113
source: 'elastic-agent.yml'
110114
mode: 0600
111115
config: true
116+
/etc/{{.BeatName}}/otel.yml:
117+
source: 'otel.yml'
118+
mode: 0600
119+
config: true
112120
/etc/{{.BeatName}}/.elastic-agent.active.commit:
113121
content: >
114122
{{ commit }}
@@ -143,6 +151,10 @@ shared:
143151
source: 'elastic-agent.yml'
144152
mode: 0600
145153
config: true
154+
'otel.yml':
155+
source: 'otel.yml'
156+
mode: 0600
157+
config: true
146158
'.elastic-agent.active.commit':
147159
content: >
148160
{{ commit }}
@@ -231,6 +243,10 @@ shared:
231243
source: 'elastic-agent.docker.yml'
232244
mode: 0600
233245
config: true
246+
'otel.yml':
247+
source: 'otel.yml'
248+
mode: 0600
249+
config: true
234250
'.elastic-agent.active.commit':
235251
content: >
236252
{{ commit }}

‎go.mod

+123-30
Original file line numberDiff line numberDiff line change
@@ -3,12 +3,12 @@ module github.com/elastic/elastic-agent
33
go 1.20
44

55
require (
6-
github.com/Microsoft/go-winio v0.6.0
6+
github.com/Microsoft/go-winio v0.6.1
77
github.com/antlr/antlr4/runtime/Go/antlr/v4 v4.0.0-20230321174746-8dcc6526cfb1
88
github.com/billgraziano/dpapi v0.4.0
99
github.com/blakesmith/ar v0.0.0-20150311145944-8bd4349a67f2
1010
github.com/cavaliercoder/go-rpm v0.0.0-20190131055624-7a9c54e3d83e
11-
github.com/cenkalti/backoff/v4 v4.1.3
11+
github.com/cenkalti/backoff/v4 v4.2.1
1212
github.com/docker/go-units v0.5.0
1313
github.com/dolmen-go/contextio v0.0.0-20200217195037-68fc5150bcd5
1414
github.com/elastic/e2e-testing v1.1.0
@@ -22,12 +22,12 @@ require (
2222
github.com/elastic/go-sysinfo v1.11.2
2323
github.com/elastic/go-ucfg v0.8.6
2424
github.com/fatih/color v1.15.0
25-
github.com/fsnotify/fsnotify v1.6.0
25+
github.com/fsnotify/fsnotify v1.7.0
2626
github.com/gofrs/flock v0.8.1
2727
github.com/gofrs/uuid v4.4.0+incompatible
28-
github.com/google/go-cmp v0.5.9
28+
github.com/google/go-cmp v0.6.0
2929
github.com/google/pprof v0.0.0-20230426061923-93006964c1fc
30-
github.com/google/uuid v1.3.0
30+
github.com/google/uuid v1.4.0
3131
github.com/gorilla/mux v1.8.0
3232
github.com/hashicorp/go-multierror v1.1.1
3333
github.com/hectane/go-acl v0.0.0-20190604041725-da78bae5fc95
@@ -39,31 +39,48 @@ require (
3939
github.com/magefile/mage v1.15.0
4040
github.com/mitchellh/gox v1.0.1
4141
github.com/mitchellh/hashstructure v1.1.0
42-
github.com/mitchellh/mapstructure v1.5.0
42+
github.com/mitchellh/mapstructure v1.5.1-0.20220423185008-bf980b35cac4
4343
github.com/oklog/ulid v1.3.1
44+
github.com/open-telemetry/opentelemetry-collector-contrib/exporter/fileexporter v0.89.0
45+
github.com/open-telemetry/opentelemetry-collector-contrib/processor/attributesprocessor v0.89.0
46+
github.com/open-telemetry/opentelemetry-collector-contrib/processor/resourceprocessor v0.89.0
47+
github.com/open-telemetry/opentelemetry-collector-contrib/processor/transformprocessor v0.89.0
48+
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/filelogreceiver v0.89.0
4449
github.com/otiai10/copy v1.14.0
4550
github.com/pierrre/gotestcover v0.0.0-20160517101806-924dca7d15f0
4651
github.com/pkg/errors v0.9.1
4752
github.com/rs/zerolog v1.27.0
4853
github.com/schollz/progressbar/v3 v3.13.1
49-
github.com/shirou/gopsutil/v3 v3.23.2
50-
github.com/sirupsen/logrus v1.9.0
51-
github.com/spf13/cobra v1.7.0
54+
github.com/shirou/gopsutil/v3 v3.23.10
55+
github.com/sirupsen/logrus v1.9.3
56+
github.com/spf13/cobra v1.8.0
5257
github.com/stretchr/testify v1.8.4
5358
github.com/tsg/go-daemon v0.0.0-20200207173439-e704b93fd89b
5459
go.elastic.co/apm/module/apmgorilla v1.15.0
5560
go.elastic.co/ecszap v1.0.1
5661
go.elastic.co/go-licence-detector v0.5.0
57-
go.uber.org/zap v1.25.0
58-
golang.org/x/crypto v0.14.0
62+
go.opentelemetry.io/collector/component v0.89.0
63+
go.opentelemetry.io/collector/confmap v0.89.0
64+
go.opentelemetry.io/collector/exporter v0.89.0
65+
go.opentelemetry.io/collector/exporter/debugexporter v0.89.0
66+
go.opentelemetry.io/collector/exporter/otlpexporter v0.89.0
67+
go.opentelemetry.io/collector/otelcol v0.89.0
68+
go.opentelemetry.io/collector/processor v0.89.0
69+
go.opentelemetry.io/collector/processor/batchprocessor v0.89.0
70+
go.opentelemetry.io/collector/processor/memorylimiterprocessor v0.89.0
71+
go.opentelemetry.io/collector/receiver v0.89.0
72+
go.opentelemetry.io/collector/receiver/otlpreceiver v0.89.0
73+
go.uber.org/zap v1.26.0
74+
golang.org/x/crypto v0.15.0
5975
golang.org/x/lint v0.0.0-20210508222113-6edffad5e616
60-
golang.org/x/sync v0.3.0
61-
golang.org/x/sys v0.13.0
62-
golang.org/x/term v0.13.0
63-
golang.org/x/text v0.13.0
76+
golang.org/x/sync v0.5.0
77+
golang.org/x/sys v0.14.0
78+
golang.org/x/term v0.14.0
79+
golang.org/x/text v0.14.0
6480
golang.org/x/time v0.3.0
65-
golang.org/x/tools v0.7.0
66-
google.golang.org/grpc v1.58.3
81+
golang.org/x/tools v0.15.0
82+
golang.org/x/tools/go/vcs v0.1.0-deprecated
83+
google.golang.org/grpc v1.59.0
6784
google.golang.org/protobuf v1.31.0
6885
gopkg.in/ini.v1 v1.67.0
6986
gopkg.in/yaml.v2 v2.4.0
@@ -77,11 +94,17 @@ require (
7794
)
7895

7996
require (
97+
contrib.go.opencensus.io/exporter/prometheus v0.4.2 // indirect
8098
github.com/Jeffail/gabs/v2 v2.6.0 // indirect
8199
github.com/StackExchange/wmi v1.2.1 // indirect
82100
github.com/akavel/rsrc v0.8.0 // indirect
101+
github.com/alecthomas/participle/v2 v2.1.0 // indirect
102+
github.com/antonmedv/expr v1.15.3 // indirect
83103
github.com/armon/go-radix v1.0.0 // indirect
104+
github.com/beorn7/perks v1.0.1 // indirect
105+
github.com/bmatcuk/doublestar/v4 v4.6.1 // indirect
84106
github.com/cavaliercoder/badio v0.0.0-20160213150051-ce5280129e9e // indirect
107+
github.com/cespare/xxhash/v2 v2.2.0 // indirect
85108
github.com/cyphar/filepath-securejoin v0.2.4 // indirect
86109
github.com/davecgh/go-spew v1.1.1 // indirect
87110
github.com/dnephin/pflag v1.0.7 // indirect
@@ -93,68 +116,139 @@ require (
93116
github.com/elastic/gosigar v0.14.2 // indirect
94117
github.com/emicklei/go-restful/v3 v3.10.1 // indirect
95118
github.com/evanphx/json-patch v5.6.0+incompatible // indirect
119+
github.com/felixge/httpsnoop v1.0.3 // indirect
96120
github.com/ghodss/yaml v1.0.0 // indirect
97-
github.com/go-logr/logr v1.2.3 // indirect
121+
github.com/go-kit/log v0.2.1 // indirect
122+
github.com/go-logfmt/logfmt v0.5.1 // indirect
123+
github.com/go-logr/logr v1.3.0 // indirect
124+
github.com/go-logr/stdr v1.2.2 // indirect
98125
github.com/go-ole/go-ole v1.2.6 // indirect
99126
github.com/go-openapi/jsonpointer v0.19.5 // indirect
100127
github.com/go-openapi/jsonreference v0.20.0 // indirect
101128
github.com/go-openapi/swag v0.22.3 // indirect
102129
github.com/gobuffalo/here v0.6.0 // indirect
130+
github.com/gobwas/glob v0.2.3 // indirect
103131
github.com/gogo/protobuf v1.3.2 // indirect
132+
github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da // indirect
104133
github.com/golang/protobuf v1.5.3 // indirect
134+
github.com/golang/snappy v0.0.4 // indirect
105135
github.com/google/gnostic v0.6.9 // indirect
106136
github.com/google/gofuzz v1.2.0 // indirect
107137
github.com/google/licenseclassifier v0.0.0-20200402202327-879cb1424de0 // indirect
108138
github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510 // indirect
139+
github.com/grpc-ecosystem/grpc-gateway/v2 v2.16.0 // indirect
109140
github.com/hashicorp/errwrap v1.1.0 // indirect
141+
github.com/iancoleman/strcase v0.3.0 // indirect
110142
github.com/imdario/mergo v0.3.13 // indirect
111143
github.com/inconshreveable/mousetrap v1.1.0 // indirect
144+
github.com/influxdata/go-syslog/v3 v3.0.1-0.20210608084020-ac565dc76ba6 // indirect
112145
github.com/jaypipes/pcidb v1.0.0 // indirect
113146
github.com/jcchavezs/porto v0.1.0 // indirect
114147
github.com/josharian/intern v1.0.0 // indirect
115148
github.com/karrick/godirwalk v1.16.1 // indirect
149+
github.com/klauspost/compress v1.17.2 // indirect
150+
github.com/knadh/koanf/maps v0.1.1 // indirect
151+
github.com/knadh/koanf/providers/confmap v0.1.0 // indirect
152+
github.com/knadh/koanf/v2 v2.0.1 // indirect
153+
github.com/leodido/ragel-machinery v0.0.0-20181214104525-299bdde78165 // indirect
116154
github.com/lufia/plan9stats v0.0.0-20211012122336-39d0f177ccd0 // indirect
117155
github.com/mailru/easyjson v0.7.7 // indirect
118156
github.com/markbates/pkger v0.17.0 // indirect
119157
github.com/mattn/go-colorable v0.1.13 // indirect
120158
github.com/mattn/go-isatty v0.0.17 // indirect
121159
github.com/mattn/go-runewidth v0.0.15 // indirect
160+
github.com/matttproud/golang_protobuf_extensions/v2 v2.0.0 // indirect
122161
github.com/mitchellh/colorstring v0.0.0-20190213212951-d06e56a500db // indirect
162+
github.com/mitchellh/copystructure v1.2.0 // indirect
123163
github.com/mitchellh/go-homedir v1.1.0 // indirect
124164
github.com/mitchellh/iochan v1.0.0 // indirect
165+
github.com/mitchellh/reflectwalk v1.0.2 // indirect
125166
github.com/moby/spdystream v0.2.0 // indirect
126167
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
127168
github.com/modern-go/reflect2 v1.0.2 // indirect
169+
github.com/mostynb/go-grpc-compression v1.2.2 // indirect
128170
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect
129171
github.com/onsi/ginkgo/v2 v2.9.0 // indirect
172+
github.com/open-telemetry/opentelemetry-collector-contrib/internal/coreinternal v0.89.0 // indirect
173+
github.com/open-telemetry/opentelemetry-collector-contrib/internal/filter v0.89.0 // indirect
174+
github.com/open-telemetry/opentelemetry-collector-contrib/internal/sharedcomponent v0.89.0 // indirect
175+
github.com/open-telemetry/opentelemetry-collector-contrib/pkg/ottl v0.89.0 // indirect
176+
github.com/open-telemetry/opentelemetry-collector-contrib/pkg/pdatautil v0.89.0 // indirect
177+
github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza v0.89.0 // indirect
130178
github.com/opencontainers/go-digest v1.0.0 // indirect
131-
github.com/opencontainers/image-spec v1.1.0-rc2 // indirect
179+
github.com/opencontainers/image-spec v1.1.0-rc5 // indirect
132180
github.com/pmezard/go-difflib v1.0.0 // indirect
133181
github.com/power-devops/perfstat v0.0.0-20210106213030-5aafc221ea8c // indirect
134-
github.com/prometheus/procfs v0.9.0 // indirect
182+
github.com/prometheus/client_golang v1.17.0 // indirect
183+
github.com/prometheus/client_model v0.5.0 // indirect
184+
github.com/prometheus/common v0.45.0 // indirect
185+
github.com/prometheus/procfs v0.11.1 // indirect
186+
github.com/prometheus/statsd_exporter v0.22.7 // indirect
135187
github.com/rivo/uniseg v0.4.4 // indirect
188+
github.com/rs/cors v1.10.1 // indirect
136189
github.com/santhosh-tekuri/jsonschema v1.2.4 // indirect
137190
github.com/sergi/go-diff v1.2.0 // indirect
138191
github.com/shirou/gopsutil v3.21.11+incompatible // indirect
192+
github.com/shoenig/go-m1cpu v0.1.6 // indirect
139193
github.com/spf13/pflag v1.0.5 // indirect
140194
github.com/stretchr/objx v0.5.0 // indirect
141-
github.com/tklauser/go-sysconf v0.3.11 // indirect
142-
github.com/tklauser/numcpus v0.6.0 // indirect
195+
github.com/tklauser/go-sysconf v0.3.12 // indirect
196+
github.com/tklauser/numcpus v0.6.1 // indirect
143197
github.com/youmark/pkcs8 v0.0.0-20201027041543-1326539a0a0a // indirect
144-
github.com/yusufpapurcu/wmi v1.2.2 // indirect
198+
github.com/yusufpapurcu/wmi v1.2.3 // indirect
145199
go.elastic.co/apm/module/apmhttp v1.15.0 // indirect
146200
go.elastic.co/apm/module/apmhttp/v2 v2.0.0 // indirect
147201
go.elastic.co/apm/v2 v2.0.0 // indirect
148202
go.elastic.co/fastjson v1.1.0 // indirect
203+
go.opencensus.io v0.24.0 // indirect
204+
go.opentelemetry.io/collector v0.89.0 // indirect
205+
go.opentelemetry.io/collector/config/configauth v0.89.0 // indirect
206+
go.opentelemetry.io/collector/config/configcompression v0.89.0 // indirect
207+
go.opentelemetry.io/collector/config/configgrpc v0.89.0 // indirect
208+
go.opentelemetry.io/collector/config/confighttp v0.89.0 // indirect
209+
go.opentelemetry.io/collector/config/confignet v0.89.0 // indirect
210+
go.opentelemetry.io/collector/config/configopaque v0.89.0 // indirect
211+
go.opentelemetry.io/collector/config/configtelemetry v0.89.0 // indirect
212+
go.opentelemetry.io/collector/config/configtls v0.89.0 // indirect
213+
go.opentelemetry.io/collector/config/internal v0.89.0 // indirect
214+
go.opentelemetry.io/collector/connector v0.89.0 // indirect
215+
go.opentelemetry.io/collector/consumer v0.89.0 // indirect
216+
go.opentelemetry.io/collector/extension v0.89.0 // indirect
217+
go.opentelemetry.io/collector/extension/auth v0.89.0 // indirect
218+
go.opentelemetry.io/collector/featuregate v1.0.0-rcv0018 // indirect
219+
go.opentelemetry.io/collector/pdata v1.0.0-rcv0018 // indirect
220+
go.opentelemetry.io/collector/semconv v0.89.0 // indirect
221+
go.opentelemetry.io/collector/service v0.89.0 // indirect
222+
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.46.0 // indirect
223+
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.45.0 // indirect
224+
go.opentelemetry.io/contrib/propagators/b3 v1.20.0 // indirect
225+
go.opentelemetry.io/otel v1.20.0 // indirect
226+
go.opentelemetry.io/otel/bridge/opencensus v0.43.0 // indirect
227+
go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetricgrpc v0.43.0 // indirect
228+
go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttp v0.43.0 // indirect
229+
go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.20.0 // indirect
230+
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.20.0 // indirect
231+
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp v1.20.0 // indirect
232+
go.opentelemetry.io/otel/exporters/prometheus v0.43.0 // indirect
233+
go.opentelemetry.io/otel/exporters/stdout/stdoutmetric v0.43.0 // indirect
234+
go.opentelemetry.io/otel/exporters/stdout/stdouttrace v1.20.0 // indirect
235+
go.opentelemetry.io/otel/metric v1.20.0 // indirect
236+
go.opentelemetry.io/otel/sdk v1.20.0 // indirect
237+
go.opentelemetry.io/otel/sdk/metric v1.20.0 // indirect
238+
go.opentelemetry.io/otel/trace v1.20.0 // indirect
239+
go.opentelemetry.io/proto/otlp v1.0.0 // indirect
149240
go.uber.org/multierr v1.11.0 // indirect
150-
golang.org/x/exp v0.0.0-20220722155223-a9213eeb770e // indirect
151-
golang.org/x/mod v0.9.0 // indirect
152-
golang.org/x/net v0.17.0 // indirect
153-
golang.org/x/oauth2 v0.10.0 // indirect
241+
golang.org/x/exp v0.0.0-20230711023510-fffb14384f22 // indirect
242+
golang.org/x/mod v0.14.0 // indirect
243+
golang.org/x/net v0.18.0 // indirect
244+
golang.org/x/oauth2 v0.12.0 // indirect
245+
gonum.org/v1/gonum v0.14.0 // indirect
154246
google.golang.org/appengine v1.6.7 // indirect
155-
google.golang.org/genproto/googleapis/rpc v0.0.0-20230711160842-782d3b101e98 // indirect
247+
google.golang.org/genproto/googleapis/api v0.0.0-20230822172742-b8732ec3820d // indirect
248+
google.golang.org/genproto/googleapis/rpc v0.0.0-20231012201019-e917dd12ba7a // indirect
156249
google.golang.org/grpc/examples v0.0.0-20220304170021-431ea809a767 // indirect
157250
gopkg.in/inf.v0 v0.9.1 // indirect
251+
gopkg.in/natefinch/lumberjack.v2 v2.2.1 // indirect
158252
howett.net/plist v1.0.0 // indirect
159253
k8s.io/kube-openapi v0.0.0-20221207184640-f3cff1453715 // indirect
160254
sigs.k8s.io/json v0.0.0-20221116044647-bc3834ca7abd // indirect
@@ -163,7 +257,7 @@ require (
163257
)
164258

165259
require (
166-
github.com/hashicorp/go-version v1.2.1 // indirect
260+
github.com/hashicorp/go-version v1.6.0 // indirect
167261
github.com/json-iterator/go v1.1.12 // indirect
168262
github.com/onsi/gomega v1.27.3 // indirect
169263
go.elastic.co/apm v1.15.0
@@ -176,7 +270,6 @@ replace (
176270
github.com/Shopify/sarama => github.com/elastic/sarama v1.19.1-0.20220310193331-ebc2b0d8eef3
177271
github.com/dop251/goja => github.com/andrewkroh/goja v0.0.0-20190128172624-dd2ac4456e20
178272
github.com/dop251/goja_nodejs => github.com/dop251/goja_nodejs v0.0.0-20171011081505-adff31b136e6
179-
github.com/fsnotify/fsnotify => github.com/adriansr/fsnotify v1.4.8-0.20211018144411-a81f2b630e7c
180273
github.com/tonistiigi/fifo => github.com/containerd/fifo v0.0.0-20190816180239-bda0ff6ed73c
181274
)
182275

0 commit comments

Comments
 (0)