[FORK][FEATURE] DQ IP: performance enhansments #272
Merged
+418
−202
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Collaborator
dmitry-gorokhov
commented
Jan 21, 2025
•
dmitry-gorokhov
commented
- allocate aux accums regs on stack
- precompute grouped src sums
- optimize pointer arithmetic
- reduce aux vecs count requred for the microkernel
dmitry-gorokhov
force-pushed
the
feature/dq_ip_accum_stack_allocation
branch
3 times, most recently
from
9960b67 to
d421730
Compare
dmitry-gorokhov
force-pushed
the
feature/dq_ip_accum_stack_allocation
branch
from
bc4e68a to
b145489
Compare
dmitry-gorokhov
changed the title
dmitry-gorokhov
changed the title
usstq
reviewed
Feb 7, 2025
- allocate aux accums regs on stack - precompute grouped src sums - optimize pointer arithmetic - reduce aux vecs count requred for the microkernel
dmitry-gorokhov
force-pushed
the
feature/dq_ip_accum_stack_allocation
branch
from
b145489 to
157fb81
Compare
github-merge-queue bot
pushed a commit
to openvinotoolkit/openvino
that referenced
this pull request
Feb 10, 2025
### Details: - Enables FC dynamic quantization on systems with AVX2 ISA - Speed-ups for LLMs with asym compressions on LNL vs master: 2x in average for prompt processing and 1.5x in average for generation stage. - OneDNN fork PR: openvinotoolkit/oneDNN#272 ### Tickets: - [CVS-146064](https://jira.devtools.intel.com/browse/CVS-146064) <- perf numbers - [CVS-134037](https://jira.devtools.intel.com/browse/CVS-134037) - [CVS-134034](https://jira.devtools.intel.com/browse/CVS-134034)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.