Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor(meta): optimize Hummock version delta deletion #20114

Open
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

zwang28
Copy link
Contributor

@zwang28 zwang28 commented Jan 12, 2025

I hereby agree to the terms of the RisingWave Labs, Inc. Contributor License Agreement.

What's changed and what's your intention?

This PR removes the unnecessary clone during delete_version_deltas. The clone may has significant impact on the performance if the number of hummock_version_deltas is large. In the worst case, deleting hummock_version_deltas can be slower than adding new ones, leading to a backlog of hummock_version_deltas.

This PR also add logs to help monitor the progresss of Hummock version redo during startup. It's helpful for troubleshooting when there're lots of
hummock_version_deltas and the redo is slow.

This PR also move time travel metadata GC to dedicated tokio task.

Checklist

  • I have written necessary rustdoc comments.
  • I have added necessary unit tests and integration tests.
  • I have added test labels as necessary.
  • I have added fuzzing tests or opened an issue to track them.
  • My PR contains breaking changes.
  • My PR changes performance-critical code, so I will run (micro) benchmarks and present the results.
  • My PR contains critical fixes that are necessary to be merged into the latest release.

Documentation

  • My PR needs documentation updates.
Release note

@zwang28 zwang28 force-pushed the wangzheng/refactor_redo branch from 40289af to 749eceb Compare January 12, 2025 03:04
@zwang28 zwang28 changed the title refactor(meta): configurable batch size for metadata deletion refactor(meta): optimize Hummock version delta deletion Jan 12, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant