Skip to content

Refactor TransactionLogTail#loadNewTail method #25856

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

chenjian2664
Copy link
Contributor

Refactor TransactionLogTail#loadNewTail to support loading transaction log to based on param startVersion and endVersion conditional. Previously, the method only supported forward traversal from head to tail, which posed a risk when early transaction logs were missing -- maybe due to cleanup. This refactor enables both forward and backward traversal, increasing robustness and flexibility in tail loading logic.

Additionally, remove BaseTransactionsTable#loadNewTailBackward and use the enhanced loadNewTail method.

Description

Additional context and related issues

Release notes

(x) This is not user-visible or is docs only, and no release notes are required.
( ) Release notes are required. Please propose a release note for me.
( ) Release notes are required, with the following suggested text:

## Section
* Fix some things. ({issue}`issuenumber`)

@cla-bot cla-bot bot added the cla-signed label May 23, 2025
@chenjian2664 chenjian2664 requested review from ebyhr and raunaqmorarka and removed request for ebyhr May 23, 2025 13:32
@github-actions github-actions bot added the delta-lake Delta Lake connector label May 23, 2025
@chenjian2664 chenjian2664 requested a review from findinpath May 23, 2025 13:33
Refactor TransactionLogTail#loadNewTail to support loading transaction log
to based on param `startVersion` and `endVersion` conditional.
Previously, the method only supported forward traversal from head to tail,
which posed a risk when early transaction logs were missing -- maybe due to cleanup.
This refactor enables both forward and backward traversal, increasing
robustness and flexibility in tail loading logic.

Additionally, remove `BaseTransactionsTable#loadNewTailBackward` and use the enhanced loadNewTail method.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cla-signed delta-lake Delta Lake connector
Development

Successfully merging this pull request may close these issues.

1 participant