Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor Redpanda Migrator components #3026

Open
wants to merge 18 commits into
base: main
Choose a base branch
from
+1,957 −1,031
Open

Refactor Redpanda Migrator components #3026

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
2b00f63
Do not reject messages for which we can't perform schema ID translation
mihaitodor Dec 6, 2024
8e1c7ee
Update `redpanda_migrator` input to not convert keys to string
mihaitodor Dec 8, 2024
21ac110
Add Redpanda Migrator offset metadata
mihaitodor Nov 21, 2024
beee211
Fix error message in `redpanda_migrator_offsets` input
mihaitodor Dec 8, 2024
c9280bb
Make some more changes to the integration test
mihaitodor Dec 6, 2024
b6bc0b5
Switch the `redpanda_migrator` to the new ack mechanism
mihaitodor Dec 11, 2024
dba9ab3
Check for cluster connectivity in the `kafka_franz` input
mihaitodor Dec 16, 2024
268050b
Update github.com/twmb/franz-go
mihaitodor Dec 12, 2024
4c4988f
Make sure we exit the preflight hook if the context is cancelled
mihaitodor Dec 12, 2024
0f23ff6
Cleanup `redpanda_migrator_offsets` input
mihaitodor Dec 16, 2024
6833a69
Cleanup NewFranzReaderOrderedFromConfig constructor
mihaitodor Dec 31, 2024
f29972d
Refactor Migrator integration test
mihaitodor Dec 12, 2024
e5d5fdc
Fix Migrator integration test
mihaitodor Dec 16, 2024
9ff2b99
Add labels to Redpanda Migrator bundle subcomponents
mihaitodor Dec 31, 2024
708b33d
Don't emit tombstone messages from redpanda_migrator input
mihaitodor Dec 31, 2024
0683073
Update info.csv
mihaitodor Dec 12, 2024
e0838a5
Update changelog
mihaitodor Dec 12, 2024
5ff42a4
Address review feedback
mihaitodor Jan 4, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 11 additions & 1 deletion CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,16 @@ All notable changes to this project will be documented in this file.
- The `redpanda_migrator` input no longer converts the kafka key to string. (@mihaitodor)
- Field `multi_header` for the `redpanda_migrator` input is now deprecated. (@mihaitodor)

### Added

- New `redpanda_migrator_offsets` input. (@mihaitodor)
- Fields `offset_topic`, `offset_group`, `offset_partition`, `offset_commit_timestamp` and `offset_metadata` added to the `redpanda_migrator_offsets` output. (@mihaitodor)
- Fields `kafka_key` and `max_in_flight` for the `redpanda_migrator_offsets` output are now deprecated. (@mihaitodor)

### Changed

- The `kafka_key` and `max_in_flight` fields of the `redpanda_migrator_offsets` output have been deprecated.

## 4.43.0 - 2024-12-05

### Changed
Expand All @@ -54,7 +64,7 @@ All notable changes to this project will be documented in this file.

- Add support for `spanner` driver to SQL plugins. (@yufeng-deng)
- Add support for complex database types (JSONB, TEXT[], INET, TSVECTOR, TSRANGE, POINT, INTEGER[]) for `pg_stream` input. (@le-vlad)
- Add support for Parquet files to `bigquery` output (@rockwotj)
- Add support for Parquet files to `bigquery` output. (@rockwotj)
- (Benthos) New `exists` operator added to the `cache` processor. (@mihaitodor)
- New CLI flag `redpanda-license` added as an alternative way to specify a Redpanda license. (@Jeffail)

Expand Down
Loading