Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Make benchmarks more consistent #3295

Merged
merged 10 commits into from
Mar 5, 2024
Merged

Make benchmarks more consistent #3295

merged 10 commits into from
Mar 5, 2024
+109 −77

Conversation

michel-laterman
Copy link
Contributor

@michel-laterman michel-laterman commented Feb 21, 2024 •
edited
Loading

What is the problem this PR solves?

Automated benchstats comparison is inconsistent

How does this PR solve the problem?

Call b.ResetTimer() directly before main benchmarks are run in order to ignore any overhead in setting up the benchmarks
Restructure some of the benchmarks for more consistent performance.
Use a nop logger is some benchmarks.
Run benchstat after the individual benchmark jobs to get a summery of the benchmarks for the pr/main.

Related issues

@michel-laterman michel-laterman added the Team:Fleet Label for the Fleet team label Feb 21, 2024
@michel-laterman
Consistently reset benchmark timers
Loading
Loading status checks…
b15648d
apmmachine and others added 3 commits March 4, 2024 13:13
@apmmachine @michel-laterman
chore: Update snapshot.yml (elastic#3311)
Loading
Loading status checks…
171fd15 
Made with ❤️️ by updatecli

Co-authored-by: apmmachine <apmmachine@users.noreply.github.com>
@michel-laterman
Use fleet-server binary version for cloude2e images (elastic#3314)
Loading
Loading status checks…
959667f 
Use the fleet-server binary version with snapshot flag when building the
cloude2e test image instead of the stack version so fleet-server builds
will not fail on version bumps.
@michel-laterman
change benchmarks
Loading
Loading status checks…
75451c7
@mergify Mergify
Copy link
Contributor

mergify bot commented Mar 4, 2024

This pull request is now in conflicts. Could you fix it @michel-laterman? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b benchmarks upstream/benchmarks
git merge upstream/main
git push upstream benchmarks

@elastic-sonarqube Elastic SonarQube
Copy link

Quality Gate passed Quality Gate passed

Kudos, no new issues were introduced!

0 New issues
0 Security Hotspots
No Coverage information No data about Coverage
0.0% 0.0% Duplication on New Code

See analysis details on SonarQube

@michel-laterman michel-laterman changed the title Consistently reset benchmark timers Make benchmarks more consistent Mar 5, 2024
@michel-laterman michel-laterman marked this pull request as ready for review March 5, 2024 11:23
@michel-laterman michel-laterman requested a review from a team as a code owner March 5, 2024 11:23
juliaElastic
juliaElastic approved these changes Mar 5, 2024
View reviewed changes
Copy link
Contributor

@juliaElastic juliaElastic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@michel-laterman michel-laterman merged commit afb4729 into elastic:main Mar 5, 2024
8 checks passed
@michel-laterman michel-laterman deleted the benchmarks branch March 5, 2024 14:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@juliaElastic juliaElastic

Assignees
No one assigned
Labels
Team:Fleet Label for the Fleet team
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[Performance] Improve fleet server benchmarks
3 participants
@michel-laterman @juliaElastic @apmmachine