[Epic] A collection of dynamic filtering related items

[alamb]

Is your feature request related to a problem or challenge?

This is a collection of various items related to "dynamic filtering"

Roughly speaking dynamic filters are filters who values are known during runtime but not planning (for example from the values of topk)

Here is some more background:

• Snowflake blog: Reimagining Top-K Aggregation Optimization at Snowflake
• Dynamic Filters in Starburst,

Related items

• Dynamic pruning filters from TopK state (optimize ORDER BY LIMIT queries) #15037
• Push Dynamic Join Predicates into Scan ("Sideways Information Passing", etc) #7955
• Blog post about TopK filter pushdown #15513
• Remove ParquetSource::pruning_predicate #15534

[adriangb]

Adding #15534

[adriangb]

I've started breaking out the work:

• parquet reader: move pruning predicate creation from ParquetSource to ParquetOpener #15561
• ExecutionPlan: add APIs for filter pushdown & optimizer rule to apply them #15566
• Introduce DynamicFilterSource and DynamicPhysicalExpr #15568 (may want to wait for ExecutionPlan: add APIs for filter pushdown & optimizer rule to apply them #15566)
• PR to replace the current filter pushdown via ListingTable w/ the new filter pushdown APIs by implementing them for ParquetSource
• PR to implement dynamic filter pushdown for TopK