Skip to content

[FLINK-37299][Connector/Pulsar] Flink stateless startup cannot contin… #103

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

yebai1105
Copy link

Purpose of the change

Controls whether Flink continues to consume from the location recorded by the consumer group or from the location specified by StartCursor when it starts from stateless.

1、If 'pulsar.source.resetSubscriptionCursor' = 'true', each time a Flink task is restarted, it will consume according to the consumption location specified by the StartCursor configuration.
2、If 'pulsar.source.resetSubscriptionCursor' = 'fasle', no matter what value 'StartCursor' is set to, each restart will start consuming from the location recorded by the consumer group.

Brief change log

The value of pulsar.source.resetSubscriptionCursor determines whether Flink continues to consume from the location recorded by the consumer group or from the location specified by StartCursor when it starts from stateless.

Verifying this change

This change is a minor change and don't have any tests.

Significant changes

(Please check any boxes [x] if the answer is "yes". You can first publish the PR and check them afterwards, for
convenience.)

  • Dependencies have been added or upgraded
  • Public API has been changed (Public API is any class annotated with @Public(Evolving))
  • Serializers have been changed
  • New feature has been introduced
    • If yes, how is this documented? (not applicable / docs / JavaDocs / not documented)

…ue consuming from the current location of the consumer group
Copy link

boring-cyborg bot commented Feb 11, 2025

Thanks for opening this pull request! Please check out our contributing guidelines. (https://flink.apache.org/contributing/how-to-contribute.html)

@thomasg19930417
Copy link

@yebai1105 If the StartCursor for consumption is set to "earliest", will there be any issues just by changing this part? Looking at the code, there is no other place where the cursor is set except here. It seems that the "subscriptionInitialPosition" needs to be set when creating the consumer in the PulsarPartitionSplitReader.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants