storage/sources/postgres: support choosing a subset of columns #29689
benesch
started this conversation in
Feature Requests
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Overview
When creating PostgreSQL sources, allow users to replicate only a subset of columns for each table.
Details
Since PostgreSQL v15, PostgreSQL has supported restricting a publication to just a subset of columns on each table.
This might just work out of the box with our existing PostgreSQL source ... but I think it's more likely that this situation causes an immediate replication failure because we'll include all columns from the base table and do a snapshot including all columns, and then complain when those columns aren't present in the replication stream.
Supporting this probably isn't too much work. When someone creates a PostgreSQL source or adds a new table to an existing PostgreSQL source, we just need to teach Materialize to check whether the PostgreSQL server is v15+ and if so, exclude any columns from the table that are not present in the publication.
Beta Was this translation helpful? Give feedback.
All reactions