Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

encoding-free in-memory row representation #20017

Open
fuyufjh opened this issue Jan 3, 2025 · 1 comment
Open

encoding-free in-memory row representation #20017

fuyufjh opened this issue Jan 3, 2025 · 1 comment
Assignees
@BugenZhao
Labels
type/feature
Milestone
release-2.3

Comments

@fuyufjh
Copy link
Member

fuyufjh commented Jan 3, 2025 •
edited by lmatz
Loading

According to CPU profiling of a Join test case, a large portion of CPU usage was used to decode/encode datum in the CompactedRow.

As we know, CompactedRow was close to storage formats instead of in-memory format. The idea is, if we can design a new encoding-free in-memory row representation that is close to the StreamChunk or Datum, then we are able to save the cost of decoding & encoding, but rather directly use the reference to do any datum operation.

Context in Slack thread: https://risingwave-labs.slack.com/archives/C034TRPKN1F/p1733731711877629?thread_ts=1733464753.437989&cid=C034TRPKN1F
https://risingwave-labs.slack.com/archives/C034TRPKN1F/p1736135668504179
@github-actions github-actions bot added this to the release-2.3 milestone Jan 3, 2025
@BugenZhao BugenZhao self-assigned this Jan 6, 2025
@BugenZhao
Copy link
Member

I'd like to help investigate.

@lmatz lmatz mentioned this issue Jan 6, 2025
Open
@BugenZhao BugenZhao mentioned this issue Jan 6, 2025
Merged
8 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@BugenZhao BugenZhao

Labels
type/feature
Projects
None yet
Milestone
release-2.3
Development

No branches or pull requests
2 participants
@fuyufjh @BugenZhao