Supported Spark Operators#
The following Spark operators are currently replaced with native versions. Query stages that contain any operators not supported by Comet will fall back to regular Spark execution.
Operator |
Spark-Compatible? |
Compatibility Notes |
|---|---|---|
BatchScanExec |
Yes |
Supports Parquet files and Apache Iceberg Parquet scans. See the Comet Compatibility Guide for more information. |
BroadcastExchangeExec |
Yes |
|
BroadcastHashJoinExec |
Yes |
|
ExpandExec |
Yes |
|
FileSourceScanExec |
Yes |
Supports Parquet files. See the Comet Compatibility Guide for more information. |
FilterExec |
Yes |
|
GlobalLimitExec |
Yes |
|
HashAggregateExec |
Yes |
|
LocalLimitExec |
Yes |
|
ObjectHashAggregateExec |
Yes |
Supports a limited number of aggregates, such as |
ProjectExec |
Yes |
|
ShuffleExchangeExec |
Yes |
|
ShuffledHashJoinExec |
Yes |
|
SortExec |
Yes |
|
SortMergeJoinExec |
Yes |
|
UnionExec |
Yes |
|
WindowExec |
Yes |