Expression Compatibility#
Comet supports multiple Apache Spark versions, and the set of accelerated expressions and their compatibility notes can differ between them. Pick the page that matches the Spark version you are running:
Expressions that are not 100% Spark-compatible fall back to Spark by default, except those
with a JVM codegen-dispatch path, which stay in Comet’s native pipeline and match Spark
exactly. Set spark.comet.expression.EXPRNAME.allowIncompatible=true, where EXPRNAME is
the Spark expression class name, to run Comet’s faster native implementation despite its
differences from Spark. See the
Comet Supported Expressions Guide for more information on this
configuration setting.