Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(host_metrics source): Add additional cgroup memory metrics #22153

Merged
merged 2 commits into from
Jan 11, 2025

Conversation

nionata
Copy link
Contributor

@nionata nionata commented Jan 10, 2025

Summary

Add active and inactive metrics for anon and file memory to the cgroup collector. These additional metrics allow you to better understand the existing cgroup memory metrics. The new metrics follow reverse domain name notation. I opted to do this over keep the ordering in the memory.stat file to increase querying ergonomics.

Change Type

  • Bug fix
  • New feature
  • Non-functional (chore, refactoring, docs)
  • Performance

Is this a breaking change?

  • Yes
  • No

How did you test this PR?

cargo fmt

cargo clippy --all-targets
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.87s

cargo test
test result: ok. 1628 passed; 0 failed; 3 ignored; 0 measured; 0 filtered out; finished in 21.40s

     Running tests/e2e/mod.rs (target/debug/deps/e2e-9fbb4963c350fe5f)

./scripts/check_changelog_fragments.sh
validating 'add_active_and_inactive_cgroup_memory_metrics.feature.md'
changelog additions are valid.
make environment CONTAINER_TOOL="podman"

/git/vectordotdev/vector# mkdir /etc/vector && cat << EOF > /etc/vector/vector.yaml
> sources:
>   in:
>     type: "host_metrics"
>     collectors:
>       - "cgroups"
>     cgroups:
>       base: ""
> 
> sinks:
>   out:
>     inputs:
>       - "in"
>     type: "console"
>     encoding:
>       codec: "native_json"
> EOF

/git/vectordotdev/vector# cargo run | jq '.metric.name, .metric.gauge.value'
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.35s
     Running `target/debug/vector`
2025-01-10T01:54:02.356851Z  INFO vector::app: Log level is enabled. level="info"
2025-01-10T01:54:02.359020Z  INFO vector::app: Loading configs. paths=["/etc/vector/vector.yaml"]
2025-01-10T01:54:02.364879Z  INFO source{component_kind="source" component_id=in component_type=host_metrics}: vector::sources::host_metrics: PROCFS_ROOT is unset. Using default '/proc' for procfs root.
2025-01-10T01:54:02.364942Z  INFO source{component_kind="source" component_id=in component_type=host_metrics}: vector::sources::host_metrics: SYSFS_ROOT is unset. Using default '/sys' for sysfs root.
2025-01-10T01:54:02.365956Z  INFO vector::topology::running: Running healthchecks.
2025-01-10T01:54:02.366216Z  INFO vector::topology::builder: Healthcheck passed.
2025-01-10T01:54:02.366282Z  INFO vector: Vector has started. debug="true" version="0.44.0" arch="x86_64" revision=""
2025-01-10T01:54:02.366326Z  INFO vector::app: API is disabled, enable by setting `api.enabled` to `true` and use commands like `vector top`.
"cgroup_cpu_usage_seconds_total"
null
"cgroup_cpu_user_seconds_total"
null
"cgroup_cpu_system_seconds_total"
null
"cgroup_memory_current_bytes"
11895951360
"cgroup_memory_anon_bytes"
25882624
"cgroup_memory_file_bytes"
11654889472
"cgroup_memory_anon_active_bytes"
20480
"cgroup_memory_anon_inactive_bytes"
25440256
"cgroup_memory_file_active_bytes"
542707712
"cgroup_memory_file_inactive_bytes"
11112181760

Does this PR include user facing changes?

  • Yes. Please add a changelog fragment based on our guidelines.
  • No. A maintainer will apply the "no-changelog" label to this PR.

Checklist

  • Please read our Vector contributor resources.
  • If this PR introduces changes Vector dependencies (modifies Cargo.lock), please
    run dd-rust-license-tool write to regenerate the license inventory and commit the changes (if any). More details here.

References

https://docs.kernel.org/admin-guide/cgroup-v2.html

@nionata nionata requested a review from a team as a code owner January 10, 2025 00:56
@nionata nionata changed the title feat(host_metrics source): Add additional cgroup memory stats feat(host_metrics source): Add additional cgroup memory metrics Jan 10, 2025
@github-actions github-actions bot added the domain: sources Anything related to the Vector's sources label Jan 10, 2025
@nionata nionata force-pushed the nji/cgroup-mem-stat branch 2 times, most recently from ccb0a46 to 457ff2e Compare January 10, 2025 01:31
@nionata nionata requested review from a team as code owners January 10, 2025 01:31
@github-actions github-actions bot added the domain: external docs Anything related to Vector's external, public documentation label Jan 10, 2025
@nionata nionata force-pushed the nji/cgroup-mem-stat branch from 457ff2e to 4d40376 Compare January 10, 2025 01:33
@nionata nionata force-pushed the nji/cgroup-mem-stat branch from 4d40376 to d961b21 Compare January 10, 2025 01:35
@jhgilbert jhgilbert self-assigned this Jan 10, 2025
@nionata nionata requested a review from jhgilbert January 10, 2025 18:13
@nionata
Copy link
Contributor Author

nionata commented Jan 10, 2025

Thanks for the stamp @jhgilbert! Is there anything else y'all need from me to get this merged?

Copy link
Member

@pront pront left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @nionata

@pront pront added this pull request to the merge queue Jan 10, 2025
Merged via the queue into vectordotdev:master with commit f0fec05 Jan 11, 2025
55 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
domain: external docs Anything related to Vector's external, public documentation domain: sources Anything related to the Vector's sources
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants