cnvm: Delete old snapshots on startup #3143
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
orestisfl
added a commit
to orestisfl/cloudbeat
that referenced
this pull request
Mar 25, 2025
Pulling change out of elastic#3143 to ease reviewing. This is the result of running ```shell hermit upgrade mockery just validate-mocks ``` Reason is that old mockery complains about go 1.22's syntax ```go for i := range 10 { ... } ```
Merged
orestisfl
added a commit
that referenced
this pull request
Mar 25, 2025
Pulling change out of #3143 to ease reviewing. This is the result of running ```shell hermit upgrade mockery just validate-mocks ``` Reason is that old mockery complains about go 1.22's syntax ```go for i := range 10 { ... } ```
orestisfl
added a commit
that referenced
this pull request
Mar 25, 2025
Upgrade mockery (#3144) Pulling change out of #3143 to ease reviewing. This is the result of running ```shell hermit upgrade mockery just validate-mocks ``` Reason is that old mockery complains about go 1.22's syntax ```go for i := range 10 { ... } ``` (cherry picked from commit 7d838d1) Co-authored-by: Orestis Floros <orestis.floros@elastic.co>
2 tasks
romulets
reviewed
Mar 28, 2025
Comment on lines
+375
to
+379
| for _, tag := range snap.Tags { | ||
| if aws.ToString(tag.Key) == "Name" { | ||
| return strings.HasPrefix(aws.ToString(tag.Value), snapshotPrefix) | ||
| } | ||
| } |
There was a problem hiding this comment.
Since we are already querying already the snapshots here:
Filters: []types.Filter{
{
Name: aws.String("tag:Name"),
Values: []string{fmt.Sprintf("%s-*", snapshotPrefix)},
},
},
Do we need this piece of filtering?
There was a problem hiding this comment.
We need to filter for time anyway so the name check is there just as a sanity check.
moukoublen
reviewed
Mar 28, 2025
Comment on lines
+148
to
+150
| type contextualChan[T any] struct { | ||
| ch chan T | ||
| } |
There was a problem hiding this comment.
[ just sharing thoughts - not suggesting to change it ]
I wonder which one of "wrapper struct vs just generic functions", is leaner in that case.
eg instead of a wrapper just having something like that
func SendToChannel[T any](ctx context.Context, ch chan<- T, t T) bool
func ReadFromChannel[T any](ctx context.Context, ch <-chan T) (T, bool)There was a problem hiding this comment.
I wanted to completely avoid exposing the channel to prevent misusing it. Both can be valid approaches if direct access to the channel is needed.
moukoublen
approved these changes
Mar 28, 2025
mergify bot
pushed a commit
that referenced
this pull request
Mar 28, 2025
### Summary of your changes
Change CNVM to add a background job cleaning up old snapshots.
The logic here is:
1. Background routine is started. Since the amount of leftover snapshots can be quite high (e.g. 40k in our account), 3 cleanup workers are used
2. `IterOwnedSnapshots()` is called which goes over all snapshots, returning an iterator. Snapshots are selected if:
1. They are more than 48 hours old
2. They have a tag with key "Name" and value starting with "elastic-vulnerability"
3. They are "self-owned"
3. Cleanup returns. On context cancelled, no extra grace period is added so the process doesn't block restarts/shutdown.
### Screenshot/Data
Deleted 1178 snapshots from `2025-03-25T16:17:08.384Z` to `2025-03-25T16:20:19.504Z`
### Related Issues
Closes #3105
Closes https://github.com/elastic/sdh-security-team/issues/1168
### Checklist
- [x] I have added tests that prove my fix is effective or that my feature works
- [x] I have added the necessary README/documentation (if appropriate)
#### Introducing a new rule?
No
(cherry picked from commit 8860b9d)
2 tasks
orestisfl
added a commit
that referenced
this pull request
Mar 28, 2025
cnvm: Delete old snapshots on startup (#3143) ### Summary of your changes Change CNVM to add a background job cleaning up old snapshots. The logic here is: 1. Background routine is started. Since the amount of leftover snapshots can be quite high (e.g. 40k in our account), 3 cleanup workers are used 2. `IterOwnedSnapshots()` is called which goes over all snapshots, returning an iterator. Snapshots are selected if: 1. They are more than 48 hours old 2. They have a tag with key "Name" and value starting with "elastic-vulnerability" 3. They are "self-owned" 3. Cleanup returns. On context cancelled, no extra grace period is added so the process doesn't block restarts/shutdown. ### Screenshot/Data Deleted 1178 snapshots from `2025-03-25T16:17:08.384Z` to `2025-03-25T16:20:19.504Z` ### Related Issues Closes #3105 Closes https://github.com/elastic/sdh-security-team/issues/1168 ### Checklist - [x] I have added tests that prove my fix is effective or that my feature works - [x] I have added the necessary README/documentation (if appropriate) #### Introducing a new rule? No (cherry picked from commit 8860b9d) Co-authored-by: Orestis Floros <orestis.floros@elastic.co>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary of your changes
Change CNVM to add a background job cleaning up old snapshots.
The logic here is:
IterOwnedSnapshots()is called which goes over all snapshots, returning an iterator. Snapshots are selected if:Screenshot/Data
Deleted 1178 snapshots from
2025-03-25T16:17:08.384Zto2025-03-25T16:20:19.504ZRelated Issues
Closes #3105
Closes https://github.com/elastic/sdh-security-team/issues/1168
Checklist
Introducing a new rule?
No