Expression Compatibility

Expressions that are not 100% Spark-compatible will fall back to Spark by default and can be enabled by setting spark.comet.expression.EXPRNAME.allowIncompatible=true, where EXPRNAME is the Spark expression class name. See the Comet Supported Expressions Guide for more information on this configuration setting.

Compatibility notes are grouped by expression category:

• Aggregate Expressions
• Array Expressions
• Date/Time Expressions
• Map Expressions
• Math Expressions
• Miscellaneous Expressions
• String Expressions
• Struct Expressions
• Cast