Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Process Messages in DQL (Dead Letter Queue) #226

Open
hekike opened this issue Sep 1, 2023 · 0 comments
Open

Process Messages in DQL (Dead Letter Queue) #226

hekike opened this issue Sep 1, 2023 · 0 comments

Comments

@hekike
Copy link
Contributor

hekike commented Sep 1, 2023

When Kafka Connect ClickHouse gets a malformed message, it sends the entire batch to DQL.
It means the DQL topic can have bad and good messages mixed. Ideally, OpenMeter would have a secondary consumer for DQL, which runs validation against those messages and tries to re-process the ones that it finds the corresponding meter and matches the validator.

Related to: #225

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant