Upgrade Guides

DataFusion 54.0.0

Note: DataFusion 54.0.0 has not been released yet. The information provided in this section pertains to features and changes that have already been merged to the main branch and are awaiting release in this version.

ExecutionPlan::apply_expressions is now a required method

apply_expressions has been added as a required method on the ExecutionPlan trait (no default implementation). The same applies to the FileSource and DataSource traits. Any custom implementation of these traits must now implement apply_expressions.

Who is affected:

• Users who implement custom ExecutionPlan nodes

• Users who implement custom FileSource or DataSource sources

Migration guide:

Add apply_expressions to your implementation. Call f on each top-level PhysicalExpr your node owns, using visit_sibling to correctly propagate TreeNodeRecursion:

Node with no expressions:

fn apply_expressions(
    &self,
    _f: &mut dyn FnMut(&dyn PhysicalExpr) -> Result<TreeNodeRecursion>,
) -> Result<TreeNodeRecursion> {
    Ok(TreeNodeRecursion::Continue)
}

Node with a single expression:

fn apply_expressions(
    &self,
    f: &mut dyn FnMut(&dyn PhysicalExpr) -> Result<TreeNodeRecursion>,
) -> Result<TreeNodeRecursion> {
    f(self.predicate.as_ref())
}

Node with multiple expressions:

fn apply_expressions(
    &self,
    f: &mut dyn FnMut(&dyn PhysicalExpr) -> Result<TreeNodeRecursion>,
) -> Result<TreeNodeRecursion> {
    let mut tnr = TreeNodeRecursion::Continue;
    for expr in &self.expressions {
        tnr = tnr.visit_sibling(|| f(expr.as_ref()))?;
    }
    Ok(tnr)
}

Node whose only expressions are in output_ordering() (e.g. a synthetic test node with no owned expression fields):

fn apply_expressions(
    &self,
    f: &mut dyn FnMut(&dyn PhysicalExpr) -> Result<TreeNodeRecursion>,
) -> Result<TreeNodeRecursion> {
    let mut tnr = TreeNodeRecursion::Continue;
    if let Some(ordering) = self.cache.output_ordering() {
        for sort_expr in ordering {
            tnr = tnr.visit_sibling(|| f(sort_expr.expr.as_ref()))?;
        }
    }
    Ok(tnr)
}