Status¶
General¶
SQL Parser
SQL Query Planner
Query Optimizer
Constant folding
Join Reordering
Limit Pushdown
Projection push down
Predicate push down
Type coercion
Parallel query execution
SQL Support¶
Projection (
SELECT
)Filter (
WHERE
)Filter post-aggregate (
HAVING
)Sorting (
ORDER BY
)Limit (
LIMIT
Aggregate (
GROUP BY
)cast /try_cast
Aggregate Functions (
SUM
,MEDIAN
, and many more)Schema Queries
SHOW TABLES
SHOW COLUMNS FROM <table/view>
SHOW CREATE TABLE <view>
Basic SQL Information Schema (
TABLES
,VIEWS
,COLUMNS
)Full SQL Information Schema support
Support for nested types (
ARRAY
/LIST
andSTRUCT
. See #2326 for details)Read support
Write support
Field access (
col['field']
and [col[1]
])-
struct
Postgres JSON operators (
->
,->>
, etc.)
Subqueries
Common Table Expressions (CTE)
Set Operations (
UNION [ALL]
,INTERSECT [ALL]
,EXCEPT[ALL]
)Joins (
INNER
,LEFT
,RIGHT
,FULL
,CROSS
)Window Functions
Empty (
OVER()
)Partitioning and ordering: (
OVER(PARTITION BY <..> ORDER BY <..>)
)Custom Window (
ORDER BY time ROWS BETWEEN 2 PRECEDING AND 0 FOLLOWING)
)User Defined Window and Aggregate Functions
Catalogs
Schemas (
CREATE / DROP SCHEMA
)Tables (
CREATE / DROP TABLE
,CREATE TABLE AS SELECT
)
Data Insert
INSERT INTO
COPY .. INTO ..
CSV
JSON
Parquet
Avro
Runtime¶
Streaming Grouping
Streaming Window Evaluation
Memory limits enforced
Spilling (to disk) Sort
Spilling (to disk) Grouping
Spilling (to disk) Joins
Data Sources¶
In addition to allowing arbitrary datasources via the TableProvider
trait, DataFusion includes built in support for the following formats:
CSV
Parquet (for all primitive and nested types)
JSON
Avro
Arrow