Dynamic Filters: Passing Information Between Operators During Execution for 25x Faster Queries
This blog post introduces the query engine optimization techniques called TopK and dynamic filters. We describe the motivating use case, how these optimizations work, and how we implemented them with the Apache DataFusion community to improve performance by an order of magnitude for some query patterns.
Motivation and Results¶
The …