Miscellaneous Expressions#

CheckOverflow#

The following cases are not supported by Comet and always fall back to Spark, regardless of any allowIncompatible setting:

  • Only DecimalType is supported

Literal#

The following cases are not supported by Comet and always fall back to Spark, regardless of any allowIncompatible setting:

  • Not all data types are supported for literal values

MakeDecimal#

The following cases are not supported by Comet and always fall back to Spark, regardless of any allowIncompatible setting:

  • Only LongType input is supported

ScalarSubquery#

The following cases are not supported by Comet and always fall back to Spark, regardless of any allowIncompatible setting:

  • Not all data types are supported for scalar subquery results

SortOrder#

The following cases are not supported by Comet and always fall back to Spark, regardless of any allowIncompatible setting:

  • A nested sort key with nondefault root null placement must be deterministic: the wrapper can evaluate the key twice, for isnull(key) and its value, which could disagree

StaticInvoke#

The following cases are not supported by Comet and always fall back to Spark, regardless of any allowIncompatible setting:

  • Iceberg’s truncate(width, value) system function on a decimal column. Iceberg’s TruncateDecimal returns a Decimal that can exceed the column’s declared precision, and Spark only turns that into null when the row is materialized. An Arrow Decimal128(precision, scale) array cannot carry that intermediate, so a native kernel would null it during evaluation and change what an enclosing predicate or hash sees. Truncating -99999999999999.9999 in a decimal(18,4) column by a width of 10 is one such value: the result has 19 digits. The other truncate input types, and bucket on decimals, are unaffected.

Uuid#

The following cases are not supported by Comet and always fall back to Spark, regardless of any allowIncompatible setting:

  • uuid requires a resolved random seed