Skip to content

feat: KVBM Cache Hit events #1139

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 9 commits into
base: main
Choose a base branch
from
Open

Conversation

jthomson04
Copy link
Contributor

@jthomson04 jthomson04 commented May 20, 2025

  • Adds KV Cache hit events
  • Restructures events to include an enum of event type.
  • Refactors event management logic to accept a list of event managers, instead of a single one
  • Also includes a PoC of cache-hit-aware block management, where blocks with more cache hits are prioritized for retention and offloading

Copy link

copy-pr-bot bot commented May 22, 2025

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@jthomson04 jthomson04 force-pushed the jthomson04/cache-hit-events branch from 97bc79b to 8dc5d67 Compare May 23, 2025 15:51
@jthomson04 jthomson04 marked this pull request as draft May 23, 2025 16:34
@jthomson04 jthomson04 force-pushed the jthomson04/cache-hit-events branch from c7e8427 to 9b5c7ae Compare May 23, 2025 21:58
@jthomson04 jthomson04 marked this pull request as ready for review May 23, 2025 21:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant