datafusion.substrait¶
This module provides support for using substrait with datafusion.
For additional information about substrait, see https://substrait.io/ for more information about substrait.
Classes¶
Generates a logical plan from a substrait plan. |
|
A class representing an encodable substrait plan. |
|
Generates substrait plans from a logical plan. |
|
Provides the |
Module Contents¶
- class datafusion.substrait.Consumer¶
Generates a logical plan from a substrait plan.
- static from_substrait_plan(ctx: datafusion.context.SessionContext, plan: Plan) datafusion._internal.LogicalPlan ¶
Convert a Substrait plan to a DataFusion LogicalPlan.
- Parameters:
ctx – SessionContext to use.
plan – Substrait plan to convert.
- Returns:
LogicalPlan.
- class datafusion.substrait.Plan(plan: datafusion._internal.substrait.Plan)¶
A class representing an encodable substrait plan.
Create a substrait plan.
The user should not have to call this constructor directly. Rather, it should be created via
Serde
or py:class:Producer classes in this module.- encode() bytes ¶
Encode the plan to bytes.
- Returns:
Encoded plan.
- plan_internal¶
- class datafusion.substrait.Producer¶
Generates substrait plans from a logical plan.
- static to_substrait_plan(logical_plan: datafusion._internal.LogicalPlan, ctx: datafusion.context.SessionContext) Plan ¶
Convert a DataFusion LogicalPlan to a Substrait plan.
- Parameters:
logical_plan – LogicalPlan to convert.
ctx – SessionContext to use.
- Returns:
Substrait plan.
- class datafusion.substrait.Serde¶
Provides the
Substrait
serialization and deserialization.- static deserialize(path: str | pathlib.Path) Plan ¶
Deserialize a Substrait plan from a file.
- Parameters:
path – Path to read the Substrait plan from.
- Returns:
Substrait plan.
- static deserialize_bytes(proto_bytes: bytes) Plan ¶
Deserialize a Substrait plan from bytes.
- Parameters:
proto_bytes – Bytes to read the Substrait plan from.
- Returns:
Substrait plan.
- static serialize(sql: str, ctx: datafusion.context.SessionContext, path: str | pathlib.Path) None ¶
Serialize a SQL query to a Substrait plan and write it to a file.
- Parameters:
sql – SQL query to serialize.
ctx – SessionContext to use.
path – Path to write the Substrait plan to.
- static serialize_bytes(sql: str, ctx: datafusion.context.SessionContext) bytes ¶
Serialize a SQL query to a Substrait plan as bytes.
- Parameters:
sql – SQL query to serialize.
ctx – SessionContext to use.
- Returns:
Substrait plan as bytes.
- static serialize_to_plan(sql: str, ctx: datafusion.context.SessionContext) Plan ¶
Serialize a SQL query to a Substrait plan.
Args: sql: SQL query to serialize. ctx: SessionContext to use.
- Returns:
Substrait plan.