|
| 1 | +# POC Documentation |
| 2 | + |
| 3 | +I generated this repo using the [beats development guide](https://www.elastic.co/guide/en/beats/devguide/current/newbeat-generate.html). |
| 4 | +The kube-api call is based on the [k8s go-client example](https://github.com/kubernetes/client-go/tree/master/examples/in-cluster-client-configuration). |
| 5 | + |
| 6 | +The interesting files are: |
| 7 | +* `beater/cloudbeat.go` - the beats logic |
| 8 | +* `cloudbeat.yml` - the beats config |
| 9 | +* `Dockerfile` - runs the beat dockerized with debug flags |
| 10 | +* `JUSTFILE` - just commander file |
| 11 | + |
| 12 | + |
| 13 | +## Table of contents |
| 14 | +- [POC Documentation](#poc-documentation) |
| 15 | + - [Table of contents](#table-of-contents) |
| 16 | + - [Prerequisites](#prerequisites) |
| 17 | + - [Running Cloudbeat (without the agent)](#running-cloudbeat-without-the-agent) |
| 18 | + - [Clean up](#clean-up) |
| 19 | + - [Remote Debugging](#remote-debugging) |
| 20 | +- [{Beat}](#beat) |
| 21 | + - [Getting Started with {Beat}](#getting-started-with-beat) |
| 22 | + - [Requirements](#requirements) |
| 23 | + - [Init Project](#init-project) |
| 24 | + - [Build](#build) |
| 25 | + - [Run](#run) |
| 26 | + - [Test](#test) |
| 27 | + - [Update](#update) |
| 28 | + - [Cleanup](#cleanup) |
| 29 | + - [Clone](#clone) |
| 30 | + - [Packaging](#packaging) |
| 31 | + - [Build Elastic-Agent Docker with pre-packaged cloudbeat](#build-elastic-agent-docker-with-pre-packaged-cloudbeat) |
| 32 | + |
| 33 | + |
| 34 | +## Prerequisites |
| 35 | +**Please make sure that you run the following instructions within the `cloudbeat` directory.** |
| 36 | +1. [Just command runner](https://github.com/casey/just) |
| 37 | +2. Elasticsearch with the default username & password (`elastic` & `changeme`) running on the default port (`http://localhost:9200`) |
| 38 | +3. Kibana with running on the default port (`http://localhost:5601`) |
| 39 | +4. Setup the local env: |
| 40 | + |
| 41 | +```zsh |
| 42 | +cd cloudbeat & just setup-env |
| 43 | +``` |
| 44 | + |
| 45 | +## Running Cloudbeat (without the agent) |
| 46 | + |
| 47 | +Build & deploy cloudbeat: |
| 48 | + |
| 49 | +```zsh |
| 50 | +just build-deploy-cloudbeat |
| 51 | +``` |
| 52 | + |
| 53 | +To validate check the logs: |
| 54 | + |
| 55 | +```zsh |
| 56 | +kubectl logs -f --selector="k8s-app=cloudbeat" -n kube-system |
| 57 | +``` |
| 58 | + |
| 59 | +Now go and check out the data on your Kibana! Make sure to add a kibana dataview `logs-k8s_cis.result-*` |
| 60 | + |
| 61 | +note: when changing the fields kibana will reject events dent from the cloudbeat for not matching the existing scheme. make sure to delete the index when changing the event fields in your code. |
| 62 | + |
| 63 | +### Clean up |
| 64 | + |
| 65 | +To stop this example and clean up the pod, run: |
| 66 | +```zsh |
| 67 | +kubectl delete -f deploy/k8s/cloudbeat-ds.yaml -n kube-system |
| 68 | +``` |
| 69 | +### Remote Debugging |
| 70 | + |
| 71 | +Build & Deploy remote debug docker: |
| 72 | + |
| 73 | +```zsh |
| 74 | +just build-deploy-cloudbeat-debug |
| 75 | +``` |
| 76 | + |
| 77 | +After running the pod, expose the relevant ports: |
| 78 | +```zsh |
| 79 | +kubectl port-forward ${pod-name} -n kube-system 40000:40000 8080:8080 |
| 80 | +``` |
| 81 | + |
| 82 | +The app will wait for the debugger to connect before starting |
| 83 | + |
| 84 | +```zsh |
| 85 | +just logs-cloudbeat |
| 86 | +``` |
| 87 | + |
| 88 | +Use your favorite IDE to connect to the debugger on `localhost:40000` (for example [Goland](https://www.jetbrains.com/help/go/attach-to-running-go-processes-with-debugger.html#step-3-create-the-remote-run-debug-configuration-on-the-client-computer)) |
| 89 | + |
| 90 | +# {Beat} |
| 91 | + |
| 92 | +Welcome to {Beat}. |
| 93 | + |
| 94 | +Ensure that this folder is at the following location: |
| 95 | +`${GOPATH}/src/github.com/elastic/beats/v7/cloudbeat` |
| 96 | + |
| 97 | +## Getting Started with {Beat} |
| 98 | + |
| 99 | +### Requirements |
| 100 | + |
| 101 | +* [Golang](https://golang.org/dl/) 1.7 |
| 102 | + |
| 103 | +### Init Project |
| 104 | +To get running with {Beat} and also install the |
| 105 | +dependencies, run the following command: |
| 106 | + |
| 107 | +``` |
| 108 | +make update |
| 109 | +``` |
| 110 | + |
| 111 | +It will create a clean git history for each major step. Note that you can always rewrite the history if you wish before pushing your changes. |
| 112 | + |
| 113 | +To push {Beat} in the git repository, run the following commands: |
| 114 | + |
| 115 | +``` |
| 116 | +git remote set-url origin https://github.com/elastic/beats/v7/cloudbeat |
| 117 | +git push origin master |
| 118 | +``` |
| 119 | + |
| 120 | +For further development, check out the [beat developer guide](https://www.elastic.co/guide/en/beats/libbeat/current/new-beat.html). |
| 121 | + |
| 122 | +### Build |
| 123 | + |
| 124 | +To build the binary for {Beat} run the command below. This will generate a binary |
| 125 | +in the same directory with the name cloudbeat. |
| 126 | + |
| 127 | +``` |
| 128 | +make |
| 129 | +``` |
| 130 | + |
| 131 | + |
| 132 | +### Run |
| 133 | + |
| 134 | +To run {Beat} with debugging output enabled, run: |
| 135 | + |
| 136 | +``` |
| 137 | +./cloudbeat -c cloudbeat.yml -e -d "*" |
| 138 | +``` |
| 139 | + |
| 140 | +### Test |
| 141 | + |
| 142 | +To test {Beat}, run the following command: |
| 143 | + |
| 144 | +``` |
| 145 | +make testsuite |
| 146 | +``` |
| 147 | + |
| 148 | +alternatively: |
| 149 | +``` |
| 150 | +make unit-tests |
| 151 | +make system-tests |
| 152 | +make integration-tests |
| 153 | +make coverage-report |
| 154 | +``` |
| 155 | + |
| 156 | +The test coverage is reported in the folder `./build/coverage/` |
| 157 | + |
| 158 | +### Update |
| 159 | + |
| 160 | +Each beat has a template for the mapping in elasticsearch and a documentation for the fields |
| 161 | +which is automatically generated based on `fields.yml` by running the following command. |
| 162 | + |
| 163 | +``` |
| 164 | +make update |
| 165 | +``` |
| 166 | + |
| 167 | + |
| 168 | +### Cleanup |
| 169 | + |
| 170 | +To clean {Beat} source code, run the following command: |
| 171 | + |
| 172 | +``` |
| 173 | +make fmt |
| 174 | +``` |
| 175 | + |
| 176 | +To clean up the build directory and generated artifacts, run: |
| 177 | + |
| 178 | +``` |
| 179 | +make clean |
| 180 | +``` |
| 181 | + |
| 182 | + |
| 183 | +### Clone |
| 184 | + |
| 185 | +To clone {Beat} from the git repository, run the following commands: |
| 186 | + |
| 187 | +``` |
| 188 | +mkdir -p ${GOPATH}/src/github.com/elastic/beats/v7/cloudbeat |
| 189 | +git clone https://github.com/elastic/beats/v7/cloudbeat ${GOPATH}/src/github.com/elastic/beats/v7/cloudbeat |
| 190 | +``` |
| 191 | + |
| 192 | + |
| 193 | +For further development, check out the [beat developer guide](https://www.elastic.co/guide/en/beats/libbeat/current/new-beat.html). |
| 194 | + |
| 195 | + |
| 196 | +## Packaging |
| 197 | + |
| 198 | +The beat frameworks provides tools to crosscompile and package your beat for different platforms. This requires [docker](https://www.docker.com/) and vendoring as described above. To build packages of your beat, run the following command: |
| 199 | + |
| 200 | +``` |
| 201 | +make release |
| 202 | +``` |
| 203 | + |
| 204 | +This will fetch and create all images required for the build process. The whole process to finish can take several minutes. |
| 205 | + |
| 206 | +## Build Elastic-Agent Docker with pre-packaged cloudbeat |
| 207 | + |
| 208 | + |
| 209 | +**1.Build Elastic-Agent Docker** |
| 210 | + |
| 211 | +1. Access the Elastic-Agent dir |
| 212 | +``` |
| 213 | +$ cd x-pack/elastic-agent |
| 214 | +``` |
| 215 | +2. Build & deploy the elastic-agent docker( You might need to increase docker engine resources on your docker-engine) |
| 216 | +``` |
| 217 | +$ just build-deploy-agent # It takes a while on the first execution. |
| 218 | +``` |
| 219 | +3. Once command is finished, Verify the agent is running on your machine |
| 220 | + |
| 221 | +```zsh |
| 222 | +$ kubectl get po --selector="app=elastic-agent" --all-namespaces -o wide |
| 223 | + |
| 224 | +NAMESPACE NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES |
| 225 | +kube-system elastic-agent-9fpdz 1/1 Running 0 5h8m 172.20.0.2 kind-mono-control-plane <none> <none> |
| 226 | + |
| 227 | + |
| 228 | +``` |
| 229 | + |
| 230 | +Note: Check the jusfile for all available commands for build or deploy `$ just --summary` |
| 231 | +</br> |
| 232 | + |
0 commit comments