datafusion.user_defined#

Provides the user-defined functions for evaluation of dataframes.

Attributes#

`_R`
`udaf`
`udf`
`udtf`
`udwf`

Classes#

`Accumulator`	Defines how an `AggregateUDF` accumulates values.
`AggregateUDF`	Class for performing scalar user-defined functions (UDF).
`AggregateUDFExportable`	Type hint for object that has __datafusion_aggregate_udf__ PyCapsule.
`LogicalExtensionCodecExportable`	Type hint for objects exposing `__datafusion_logical_extension_codec__`.
`PhysicalExtensionCodecExportable`	Type hint for objects exposing `__datafusion_physical_extension_codec__`.
`ScalarUDF`	Class for performing scalar user-defined functions (UDF).
`ScalarUDFExportable`	Type hint for object that has __datafusion_scalar_udf__ PyCapsule.
`TableFunction`	Class for performing user-defined table functions (UDTF).
`Volatility`	Defines how stable or volatile a function is.
`WindowEvaluator`	Evaluator class for user-defined window functions (UDWF).
`WindowUDF`	Class for performing window user-defined functions (UDF).
`WindowUDFExportable`	Type hint for object that has __datafusion_window_udf__ PyCapsule.

Functions#

`_is_pycapsule`(→ TypeGuard[_typeshed.CapsuleType])	Return `True` when `value` is a CPython `PyCapsule`.
`_wrap_session_kwarg_for_udtf`(...)	Adapt the raw internal session pyo3 object back to a Python wrapper.
`data_type_or_field_to_field`(→ pyarrow.Field)	Helper function to return a Field from either a Field or DataType.
`data_types_or_fields_to_field_list`(→ list[pyarrow.Field])	Helper function to return a list of Fields.

Module Contents#

class datafusion.user_defined.Accumulator#

Defines how an AggregateUDF accumulates values.

abstract evaluate() → pyarrow.Scalar#

Return the resultant value.

While this function template expects a PyArrow Scalar value return type, you can return any value that can be converted into a Scalar. This includes basic Python data types such as integers and strings. In addition to primitive types, we currently support PyArrow, nanoarrow, and arro3 objects in addition to primitive data types. Other objects that support the Arrow FFI standard will be given a “best attempt” at conversion to scalar objects.

abstract merge(states: list[pyarrow.Array]) → None#: Merge a set of states.

abstract state() → list[pyarrow.Scalar]#

Return the current state.

While this function template expects PyArrow Scalar values return type, you can return any value that can be converted into a Scalar. This includes basic Python data types such as integers and strings. In addition to primitive types, we currently support PyArrow, nanoarrow, and arro3 objects in addition to primitive data types. Other objects that support the Arrow FFI standard will be given a “best attempt” at conversion to scalar objects.

abstract update(*values: pyarrow.Array) → None#: Evaluate an array of values and update state.

class datafusion.user_defined.AggregateUDF(name: str, accumulator: collections.abc.Callable[[], Accumulator], input_types: list[pyarrow.DataType], return_type: pyarrow.DataType, state_type: list[pyarrow.DataType], volatility: Volatility | str)#

class datafusion.user_defined.AggregateUDF(name: str, accumulator: AggregateUDFExportable, input_types: None = ..., return_type: None = ..., state_type: None = ..., volatility: None = ...)

Class for performing scalar user-defined functions (UDF).

Aggregate UDFs operate on a group of rows and return a single value. See also ScalarUDF for operating on a row by row basis.

Instantiate a user-defined aggregate function (UDAF).

See udaf() for a convenience function and argument descriptions.

__call__(*args: datafusion.expr.Expr) → datafusion.expr.Expr#

Execute the UDAF.

This function is not typically called by an end user. These calls will occur during the evaluation of the dataframe.

__repr__() → str#: Print a string representation of the Aggregate UDF.

classmethod _from_internal(internal: datafusion._internal.AggregateUDF) → AggregateUDF#

Wrap an already-constructed internal AggregateUDF handle.

Used by SessionContext.udaf() to surface a function looked up from the session’s function registry without re-running __init__().

static from_pycapsule(func: AggregateUDFExportable | _typeshed.CapsuleType) → AggregateUDF#

Create an Aggregate UDF from AggregateUDF PyCapsule object.

This function will instantiate a Aggregate UDF that uses a DataFusion AggregateUDF that is exported via the FFI bindings.

static udaf(input_types: pyarrow.DataType | list[pyarrow.DataType], return_type: pyarrow.DataType, state_type: list[pyarrow.DataType], volatility: Volatility | str, name: str | None = None) → collections.abc.Callable[Ellipsis, AggregateUDF]#

static udaf(accum: collections.abc.Callable[[], Accumulator], input_types: pyarrow.DataType | list[pyarrow.DataType], return_type: pyarrow.DataType, state_type: list[pyarrow.DataType], volatility: Volatility | str, name: str | None = None) → AggregateUDF

static udaf(accum: AggregateUDFExportable) → AggregateUDF

static udaf(accum: _typeshed.CapsuleType) → AggregateUDF

Create a new User-Defined Aggregate Function (UDAF).

This class allows you to define an aggregate function that can be used in data aggregation or window function calls.

Usage:

As a function: udaf(accum, input_types, return_type, state_type, volatility, name).
As a decorator: @udaf(input_types, return_type, state_type, volatility, name). When using udaf as a decorator, do not pass accum explicitly.

If your Accumulator can be instantiated with no arguments, you can simply pass its type as accum. If you need to pass additional arguments to its constructor, you can define a lambda or a factory method. During runtime the Accumulator will be constructed for every instance in which this UDAF is used.

Examples

>>> import pyarrow.compute as pc
>>> from datafusion.user_defined import AggregateUDF, Accumulator, udaf
>>> class Summarize(Accumulator):
...     def __init__(self, bias: float = 0.0):
...         self._sum = pa.scalar(bias)
...     def state(self):
...         return [self._sum]
...     def update(self, values):
...         self._sum = pa.scalar(
...             self._sum.as_py() + pc.sum(values).as_py())
...     def merge(self, states):
...         self._sum = pa.scalar(
...             self._sum.as_py() + pc.sum(states[0]).as_py())
...     def evaluate(self):
...         return self._sum

Using udaf as a function:

>>> udaf1 = AggregateUDF.udaf(
...     Summarize, pa.float64(), pa.float64(),
...     [pa.float64()], "immutable")

Wrapping udaf with a function:

>>> def sum_bias_10() -> Summarize:
...     return Summarize(10.0)
>>> udaf2 = udaf(sum_bias_10, pa.float64(), pa.float64(), [pa.float64()],
...     "immutable")

Using udaf with lambda:

>>> udaf3 = udaf(lambda: Summarize(20.0), pa.float64(), pa.float64(),
...     [pa.float64()], "immutable")

Using udaf as a decorator:

>>> @AggregateUDF.udaf(
...     pa.float64(), pa.float64(),
...     [pa.float64()], "immutable")
... def udaf4():
...     return Summarize(10.0)

Apply to a dataframe:

>>> ctx = dfn.SessionContext()
>>> df = ctx.from_pydict({"a": [1.0, 2.0, 3.0]})
>>> df.aggregate([], [udaf1(col("a")).alias("total")]).collect_column(
...     "total")[0].as_py()
6.0
>>> df.aggregate([], [udaf2(col("a")).alias("total")]).collect_column(
...     "total")[0].as_py()
16.0
>>> df.aggregate([], [udaf3(col("a")).alias("total")]).collect_column(
...     "total")[0].as_py()
26.0
>>> df.aggregate([], [udaf4(col("a")).alias("total")]).collect_column(
...     "total")[0].as_py()
16.0

Parameters:

accum – The accumulator python function. Only needed when calling as a function. Skip this argument when using udaf as a decorator. If you have a Rust backed AggregateUDF within a PyCapsule, you can pass this parameter and ignore the rest. They will be determined directly from the underlying function. See the online documentation for more information.
input_types – The data types of the arguments to accum.
return_type – The data type of the return value.
state_type – The data types of the intermediate accumulation.
volatility – See Volatility for allowed values.
name – A descriptive name for the function.

Returns:

A user-defined aggregate function, which can be used in either data aggregation or window function calls.

_udaf#

property name: str#

Return the registered name of this UDAF.

For UDAFs imported via the FFI capsule protocol, this is the name the capsule itself reports — not the name argument passed to the constructor (which is ignored on the FFI path).

class datafusion.user_defined.AggregateUDFExportable#

Bases: Protocol

Type hint for object that has __datafusion_aggregate_udf__ PyCapsule.

__datafusion_aggregate_udf__() → object#

class datafusion.user_defined.LogicalExtensionCodecExportable#

Bases: Protocol

Type hint for objects exposing __datafusion_logical_extension_codec__.

__datafusion_logical_extension_codec__() → object#

class datafusion.user_defined.PhysicalExtensionCodecExportable#

Bases: Protocol

Type hint for objects exposing __datafusion_physical_extension_codec__.

__datafusion_physical_extension_codec__() → object#

class datafusion.user_defined.ScalarUDF(name: str, func: collections.abc.Callable[Ellipsis, _R], input_fields: list[pyarrow.Field], return_field: pyarrow.Field, volatility: Volatility | str)#

Class for performing scalar user-defined functions (UDF).

Scalar UDFs operate on a row by row basis. See also AggregateUDF for operating on a group of rows.

Instantiate a scalar user-defined function (UDF).

See helper method udf() for argument details.

__call__(*args: datafusion.expr.Expr) → datafusion.expr.Expr#

Execute the UDF.

This function is not typically called by an end user. These calls will occur during the evaluation of the dataframe.

__repr__() → str#: Print a string representation of the Scalar UDF.

classmethod _from_internal(internal: datafusion._internal.ScalarUDF) → ScalarUDF#

Wrap an already-constructed internal ScalarUDF handle.

Used by SessionContext.udf() to surface a function looked up from the session’s function registry without re-running __init__().

static from_pycapsule(func: ScalarUDFExportable) → ScalarUDF#

Create a Scalar UDF from ScalarUDF PyCapsule object.

This function will instantiate a Scalar UDF that uses a DataFusion ScalarUDF that is exported via the FFI bindings.

static udf(func: ScalarUDFExportable) → ScalarUDF

Create a new User-Defined Function (UDF).

This class can be used both as either a function or a decorator.

Usage:

As a function: udf(func, input_fields, return_field, volatility, name).
As a decorator: @udf(input_fields, return_field, volatility, name). When used a decorator, do not pass func explicitly.

In lieu of passing a PyArrow Field, you can pass a DataType for simplicity. When you do so, it will be assumed that the nullability of the inputs and output are True and that they have no metadata.

Parameters:

func (Callable, optional) – Only needed when calling as a function. Skip this argument when using udf as a decorator. If you have a Rust backed ScalarUDF within a PyCapsule, you can pass this parameter and ignore the rest. They will be determined directly from the underlying function. See the online documentation for more information.
input_fields (list[pa.Field | pa.DataType]) – The data types or Fields of the arguments to func. This list must be of the same length as the number of arguments.
return_field (_R) – The field of the return value from the function.
volatility (Volatility | str) – See Volatility for allowed values.
name (Optional[str]) – A descriptive name for the function.

Returns:

A user-defined function that can be used in SQL expressions, data aggregation, or window function calls.

Examples

Using udf as a function:

>>> import pyarrow.compute as pc
>>> from datafusion.user_defined import ScalarUDF
>>> def double_func(x):
...     return pc.multiply(x, 2)
>>> double_udf = ScalarUDF.udf(
...     double_func, [pa.int64()], pa.int64(),
...     "volatile", "double_it")

Using udf as a decorator:

>>> @ScalarUDF.udf([pa.int64()], pa.int64(), "volatile")
... def decorator_double_udf(x):
...     return pc.multiply(x, 3)

Apply to a dataframe:

>>> ctx = dfn.SessionContext()
>>> df = ctx.from_pydict({"x": [1, 2, 3]})
>>> df.select(double_udf(col("x")).alias("result")).to_pydict()
{'result': [2, 4, 6]}
>>> df.select(decorator_double_udf(col("x")).alias("result")).to_pydict()
{'result': [3, 6, 9]}

_udf#

property name: str#

Return the registered name of this UDF.

For UDFs imported via the FFI capsule protocol, this is the name the capsule itself reports — not the name argument passed to the constructor (which is ignored on the FFI path).

Examples

>>> import pyarrow as pa
>>> from datafusion import udf
>>> double = udf(
...     lambda arr: pa.array([(v.as_py() or 0) * 2 for v in arr]),
...     [pa.int64()],
...     pa.int64(),
...     volatility="immutable",
...     name="double",
... )
>>> double.name
'double'

class datafusion.user_defined.ScalarUDFExportable#

Bases: Protocol

Type hint for object that has __datafusion_scalar_udf__ PyCapsule.

__datafusion_scalar_udf__() → object#

class datafusion.user_defined.TableFunction(name: str, func: collections.abc.Callable[Ellipsis, Any], ctx: datafusion.SessionContext | None = None, *, with_session: bool = False)#

Class for performing user-defined table functions (UDTF).

Table functions generate new table providers based on the input expressions.

Instantiate a user-defined table function (UDTF).

Set with_session=True to have the calling SessionContext passed as a session keyword argument on each invocation. Use it inside the callback to look up registered tables, UDFs, or session configuration. When with_session is False (the default), func is invoked with the positional expression arguments only.

with_session=True is only supported for pure-Python callables. Passing it together with an FFI-exported table function (one exposing __datafusion_table_function__) raises TypeError.

Registry mutations performed through the injected session (such as registering tables or UDFs) propagate to the caller’s SessionContext because the registries are shared. Configuration changes do not propagate; the wrapper holds its own clone of the session config.

See udtf() for a convenience function and argument descriptions.

__call__(*args: datafusion.expr.Expr) → Any#: Execute the UDTF and return a table provider.

__repr__() → str#: User printable representation.

static _create_table_udf(func: collections.abc.Callable[Ellipsis, Any], name: str, *, with_session: bool = False) → TableFunction#: Create a TableFunction instance from function arguments.

static _create_table_udf_decorator(name: str | None = None, *, with_session: bool = False) → collections.abc.Callable[[collections.abc.Callable[Ellipsis, Any]], TableFunction]#: Create a decorator for a TableFunction.

static udtf(name: str, *, with_session: bool = False) → collections.abc.Callable[Ellipsis, Any]#

static udtf(func: collections.abc.Callable[Ellipsis, Any], name: str, *, with_session: bool = False) → TableFunction

Create a new User-Defined Table Function (UDTF).

Pass with_session=True to have the calling SessionContext injected as a session keyword argument on each invocation.

_udtf#

class datafusion.user_defined.Volatility#

Bases: enum.Enum

Defines how stable or volatile a function is.

When setting the volatility of a function, you can either pass this enumeration or a str. The str equivalent is the lower case value of the name (“immutable”, “stable”, or “volatile”).

__str__() → str#: Returns the string equivalent.

Immutable = 1#

An immutable function will always return the same output when given the same input.

DataFusion will attempt to inline immutable functions during planning.

Stable = 2#

Returns the same value for a given input within a single queries.

A stable function may return different values given the same input across different queries but must return the same value for a given input within a query. An example of this is the Now function. DataFusion will attempt to inline Stable functions during planning, when possible. For query select col1, now() from t1, it might take a while to execute but now() column will be the same for each output row, which is evaluated during planning.

Volatile = 3#

A volatile function may change the return value from evaluation to evaluation.

Multiple invocations of a volatile function may return different results when used in the same query. An example of this is the random() function. DataFusion can not evaluate such functions during planning. In the query select col1, random() from t1, random() function will be evaluated for each output row, resulting in a unique random value for each row.

class datafusion.user_defined.WindowEvaluator#

Evaluator class for user-defined window functions (UDWF).

It is up to the user to decide which evaluate function is appropriate.

`uses_window_frame`	`supports_bounded_execution`	`include_rank`	function_to_implement
False (default)	False (default)	False (default)	`evaluate_all`
False	True	False	`evaluate`
False	True/False	True	`evaluate_all_with_rank`
True	True/False	True/False	`evaluate`

evaluate(values: list[pyarrow.Array], eval_range: tuple[int, int]) → pyarrow.Scalar#

Evaluate window function on a range of rows in an input partition.

This is the simplest and most general function to implement but also the least performant as it creates output one row at a time. It is typically much faster to implement stateful evaluation using one of the other specialized methods on this trait.

Returns a [ScalarValue] that is the value of the window function within range for the entire partition. Argument values contains the evaluation result of function arguments and evaluation results of ORDER BY expressions. If function has a single argument, values[1..] will contain ORDER BY expression results.

evaluate_all(values: list[pyarrow.Array], num_rows: int) → pyarrow.Array#

Evaluate a window function on an entire input partition.

This function is called once per input partition for window functions that do not use values from the window frame, such as row_number(), rank(), dense_rank(), percent_rank(), cume_dist(), lead(), and lag().

It produces the result of all rows in a single pass. It expects to receive the entire partition as the value and must produce an output column with one output row for every input row.

num_rows is required to correctly compute the output in case len(values) == 0

Implementing this function is an optimization. Certain window functions are not affected by the window frame definition or the query doesn’t have a frame, and evaluate skips the (costly) window frame boundary calculation and the overhead of calling evaluate for each output row.

For example, the LAG built in window function does not use the values of its window frame (it can be computed in one shot on the entire partition with Self::evaluate_all regardless of the window defined in the OVER clause)

lag(x, 1) OVER (ORDER BY z ROWS BETWEEN 2 PRECEDING AND 3 FOLLOWING)

However, avg() computes the average in the window and thus does use its window frame.

avg(x) OVER (PARTITION BY y ORDER BY z ROWS BETWEEN 2 PRECEDING AND 3 FOLLOWING)

evaluate_all_with_rank(num_rows: int, ranks_in_partition: list[tuple[int, int]]) → pyarrow.Array#

Called for window functions that only need the rank of a row.

Evaluate the partition evaluator against the partition using the row ranks. For example, rank(col("a")) produces

a | rank
- + ----
A | 1
A | 1
C | 3
D | 4
D | 4

For this case, num_rows would be 5 and the ranks_in_partition would be called with

[
    (0,1),
    (2,2),
    (3,4),
]

The user must implement this method if include_rank returns True.

get_range(idx: int, num_rows: int) → tuple[int, int]#

Return the range for the window function.

If uses_window_frame flag is false. This method is used to calculate required range for the window function during stateful execution.

Generally there is no required range, hence by default this returns smallest range(current row). e.g seeing current row is enough to calculate window result (such as row_number, rank, etc)

Parameters:

idx: – Current index:
num_rows – Number of rows.

include_rank() → bool#: Can this function be evaluated with (only) rank?

is_causal() → bool#: Get whether evaluator needs future data for its result.

memoize() → None#

Perform a memoize operation to improve performance.

When the window frame has a fixed beginning (e.g UNBOUNDED PRECEDING), some functions such as FIRST_VALUE and NTH_VALUE do not need the (unbounded) input once they have seen a certain amount of input.

memoize is called after each input batch is processed, and such functions can save whatever they need

supports_bounded_execution() → bool#: Can the window function be incrementally computed using bounded memory?

uses_window_frame() → bool#: Does the window function use the values from the window frame?

class datafusion.user_defined.WindowUDF(name: str, func: collections.abc.Callable[[], WindowEvaluator], input_types: list[pyarrow.DataType], return_type: pyarrow.DataType, volatility: Volatility | str)#

Class for performing window user-defined functions (UDF).

Window UDFs operate on a partition of rows. See also ScalarUDF for operating on a row by row basis.

Instantiate a user-defined window function (UDWF).

See udwf() for a convenience function and argument descriptions.

__call__(*args: datafusion.expr.Expr) → datafusion.expr.Expr#

Execute the UDWF.

This function is not typically called by an end user. These calls will occur during the evaluation of the dataframe.

__repr__() → str#: Print a string representation of the Window UDF.

static _create_window_udf(func: collections.abc.Callable[[], WindowEvaluator], input_types: pyarrow.DataType | list[pyarrow.DataType], return_type: pyarrow.DataType, volatility: Volatility | str, name: str | None = None) → WindowUDF#: Create a WindowUDF instance from function arguments.

static _create_window_udf_decorator(input_types: pyarrow.DataType | list[pyarrow.DataType], return_type: pyarrow.DataType, volatility: Volatility | str, name: str | None = None) → collections.abc.Callable[[collections.abc.Callable[[], WindowEvaluator]], collections.abc.Callable[Ellipsis, datafusion.expr.Expr]]#: Create a decorator for a WindowUDF.

classmethod _from_internal(internal: datafusion._internal.WindowUDF) → WindowUDF#

Wrap an already-constructed internal WindowUDF handle.

Used by SessionContext.udwf() to surface a function looked up from the session’s function registry without re-running __init__().

static _get_default_name(func: collections.abc.Callable) → str#: Get the default name for a function based on its attributes.

static _normalize_input_types(input_types: pyarrow.DataType | list[pyarrow.DataType]) → list[pyarrow.DataType]#: Convert a single DataType to a list if needed.

static from_pycapsule(func: WindowUDFExportable) → WindowUDF#

Create a Window UDF from WindowUDF PyCapsule object.

This function will instantiate a Window UDF that uses a DataFusion WindowUDF that is exported via the FFI bindings.

static udwf(input_types: pyarrow.DataType | list[pyarrow.DataType], return_type: pyarrow.DataType, volatility: Volatility | str, name: str | None = None) → collections.abc.Callable[Ellipsis, WindowUDF]#

static udwf(func: collections.abc.Callable[[], WindowEvaluator], input_types: pyarrow.DataType | list[pyarrow.DataType], return_type: pyarrow.DataType, volatility: Volatility | str, name: str | None = None) → WindowUDF

Create a new User-Defined Window Function (UDWF).

This class can be used both as either a function or a decorator.

Usage:

As a function: udwf(func, input_types, return_type, volatility, name).
As a decorator: @udwf(input_types, return_type, volatility, name). When using udwf as a decorator, do not pass func explicitly.

Examples

>>> from datafusion.user_defined import WindowUDF, WindowEvaluator, udwf
>>> class BiasedNumbers(WindowEvaluator):
...     def __init__(self, start: int = 0):
...         self.start = start
...     def evaluate_all(self, values, num_rows):
...         return pa.array(
...             [self.start + i for i in range(num_rows)])

Using udwf as a function:

>>> udwf1 = WindowUDF.udwf(
...     BiasedNumbers, pa.int64(), pa.int64(), "immutable")
>>> def bias_10() -> BiasedNumbers:
...    return BiasedNumbers(10)
>>> udwf2 = udwf(bias_10, pa.int64(), pa.int64(), "immutable")
>>> udwf3 = udwf(
...     lambda: BiasedNumbers(20), pa.int64(), pa.int64(), "immutable"
... )

Using udwf as a decorator:

>>> @WindowUDF.udwf(pa.int64(), pa.int64(), "immutable")
... def biased_numbers():
...     return BiasedNumbers(10)

Apply to a dataframe:

>>> ctx = dfn.SessionContext()
>>> df = ctx.from_pydict({"a": [10, 20, 30]})
>>> df.select(udwf1(col("a")).alias("result")).to_pydict()
{'result': [0, 1, 2]}
>>> df.select(udwf2(col("a")).alias("result")).to_pydict()
{'result': [10, 11, 12]}
>>> df.select(udwf3(col("a")).alias("result")).to_pydict()
{'result': [20, 21, 22]}
>>> df.select(biased_numbers(col("a")).alias("result")).to_pydict()
{'result': [10, 11, 12]}

Parameters:

func – Only needed when calling as a function. Skip this argument when using udwf as a decorator. If you have a Rust backed WindowUDF within a PyCapsule, you can pass this parameter and ignore the rest. They will be determined directly from the underlying function. See the online documentation for more information.
input_types – The data types of the arguments.
return_type – The data type of the return value.
volatility – See Volatility for allowed values.
name – A descriptive name for the function.

Returns:

A user-defined window function that can be used in window function calls.

_udwf#

property name: str#

Return the registered name of this UDWF.

For UDWFs imported via the FFI capsule protocol, this is the name the capsule itself reports — not the name argument passed to the constructor (which is ignored on the FFI path).

class datafusion.user_defined.WindowUDFExportable#

Bases: Protocol

Type hint for object that has __datafusion_window_udf__ PyCapsule.

__datafusion_window_udf__() → object#

datafusion.user_defined._is_pycapsule(value: object) → TypeGuard[_typeshed.CapsuleType]#: Return True when value is a CPython PyCapsule.

datafusion.user_defined._wrap_session_kwarg_for_udtf(func: collections.abc.Callable[Ellipsis, Any]) → collections.abc.Callable[Ellipsis, Any]#

Adapt the raw internal session pyo3 object back to a Python wrapper.

The Rust call site forwards a datafusion._internal.SessionContext, but UDTF authors expect to interact with the public datafusion.SessionContext wrapper. This closure wraps the internal object once per call before delegating to func.

datafusion.user_defined.data_type_or_field_to_field(value: pyarrow.DataType | pyarrow.Field, name: str) → pyarrow.Field#: Helper function to return a Field from either a Field or DataType.

datafusion.user_defined.data_types_or_fields_to_field_list(inputs: collections.abc.Sequence[pyarrow.Field | pyarrow.DataType] | pyarrow.Field | pyarrow.DataType) → list[pyarrow.Field]#: Helper function to return a list of Fields.

datafusion.user_defined._R#

datafusion.user_defined.udaf#

datafusion.user_defined.udf#

datafusion.user_defined.udtf#

datafusion.user_defined.udwf#