Miscellaneous Expressions#

CheckOverflow#

The following cases are not supported by Comet and always fall back to Spark, regardless of any allowIncompatible setting:

Only DecimalType is supported

Literal#

The following cases are not supported by Comet and always fall back to Spark, regardless of any allowIncompatible setting:

Not all data types are supported for literal values

MakeDecimal#

The following cases are not supported by Comet and always fall back to Spark, regardless of any allowIncompatible setting:

Only LongType input is supported

ScalarSubquery#

The following cases are not supported by Comet and always fall back to Spark, regardless of any allowIncompatible setting:

Not all data types are supported for scalar subquery results

SortOrder#

The following incompatibilities cause SortOrder to fall back to Spark by default. Set spark.comet.expression.SortOrder.allowIncompatible=true to enable Comet acceleration despite these differences.

Sorting on floating-point values nested in arrays, structs, or maps is not 100% compatible with Spark when spark.comet.exec.strictFloatingPoint=true

StaticInvoke#

The following cases are not supported by Comet and always fall back to Spark, regardless of any allowIncompatible setting:

Iceberg’s truncate(width, value) system function on a decimal column. Iceberg’s TruncateDecimal returns a Decimal that can exceed the column’s declared precision, and Spark only turns that into null when the row is materialized. An Arrow Decimal128(precision, scale) array cannot carry that intermediate, so a native kernel would null it during evaluation and change what an enclosing predicate or hash sees. Truncating -99999999999999.9999 in a decimal(18,4) column by a width of 10 is one such value: the result has 19 digits. The other truncate input types, and bucket on decimals, are unaffected.

Uuid#

The following cases are not supported by Comet and always fall back to Spark, regardless of any allowIncompatible setting:

uuid requires a resolved random seed