Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[GPU] KV-cache compression micro_sdpa kernel #28004

Open
wants to merge 15 commits into
base: master
Choose a base branch
from
Open

[GPU] KV-cache compression micro_sdpa kernel #28004

wants to merge 15 commits into from
+489 −101

Conversation

sshlyapn
Copy link
Contributor

@sshlyapn sshlyapn commented Dec 10, 2024 •
edited by p-durandin
Loading

Details:

  • Added KV-cache compression support to the micro_sdpa kernel
  • Performance still needs to be adjusted

Tickets:

@sshlyapn sshlyapn added category: GPU OpenVINO GPU plugin do_not_merge labels Dec 10, 2024
@sshlyapn sshlyapn added this to the 2025.0 milestone Dec 10, 2024
@sshlyapn sshlyapn force-pushed the onednn_kv_cache_compression branch from ead8d8a to 5df2d53 Compare December 20, 2024 17:37
@p-durandin p-durandin marked this pull request as ready for review January 14, 2025 09:51
@p-durandin p-durandin requested review from a team as code owners January 14, 2025 09:51
@p-durandin p-durandin changed the title WIP: [GPU] KV-cache compression micro_sdpa kernel [GPU] KV-cache compression micro_sdpa kernel Jan 14, 2025
@moslex moslex added the priority: high High piority label Jan 16, 2025
vladimir-paramuzov
vladimir-paramuzov approved these changes Jan 16, 2025
View reviewed changes
Copy link
Contributor

@vladimir-paramuzov vladimir-paramuzov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM for initial version. Need further perf tuning

@p-durandin p-durandin enabled auto-merge January 16, 2025 10:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@vladimir-paramuzov vladimir-paramuzov vladimir-paramuzov approved these changes

Assignees
No one assigned
Labels
category: GPU OpenVINO GPU plugin Code Freeze priority: high High piority
Projects
None yet
Milestone
2025.0
Development

Successfully merging this pull request may close these issues.
4 participants
@sshlyapn @vladimir-paramuzov @moslex @p-durandin