Supported Spark OperatorsΒΆ
The following Spark operators are currently replaced with native versions. Query stages that contain any operators not supported by Comet will fall back to regular Spark execution.
Operator |
Spark-Compatible? |
Compatibility Notes |
|---|---|---|
BatchScanExec |
Yes |
Supports Parquet files and Apache Iceberg Parquet scans. See the Comet Compatibility Guide for more information. |
BroadcastExchangeExec |
Yes |
|
BroadcastHashJoinExec |
Yes |
|
ExpandExec |
Yes |
|
FileSourceScanExec |
Yes |
Supports Parquet files. See the Comet Compatibility Guide for more information. |
FilterExec |
Yes |
|
GlobalLimitExec |
Yes |
|
HashAggregateExec |
Yes |
|
LocalLimitExec |
Yes |
|
ObjectHashAggregateExec |
Yes |
Supports a limited number of aggregates, such as |
ProjectExec |
Yes |
|
ShuffleExchangeExec |
Yes |
|
ShuffledHashJoinExec |
Yes |
|
SortExec |
Yes |
|
SortMergeJoinExec |
Yes |
|
UnionExec |
Yes |
|
WindowExec |
Yes |