Package org.apache.datafusion

Class SessionContext

java.lang.Object

org.apache.datafusion.SessionContext

All Implemented Interfaces:

AutoCloseable

public final class SessionContext
extends Object
implements AutoCloseable

A DataFusion session context.

Instances are not thread-safe. Concurrent calls to any of sql(java.lang.String), registerParquet(java.lang.String, java.lang.String), or close() from different threads can produce a use-after-free on the native side. Callers must externally synchronize, or confine each context to a single thread.

Constructor Summary

Constructors

Constructor	Description
SessionContext()

Method Summary

Modifier and Type	Method	Description
static SessionContextBuilder	builder()	Start configuring a SessionContext.
void	close()
DataFrame	fromProto(byte[] planBytes)	Decode a DataFusion-Proto LogicalPlanNode and return a lazy DataFrame.
String	getOption(String key)	Read the current value of a datafusion.* config key.
DataFrame	readArrow(String path)	Read an Arrow IPC file as a DataFrame without registering it.
DataFrame	readArrow(String path, ArrowReadOptions options)	Read an Arrow IPC file as a DataFrame with the supplied ArrowReadOptions.
DataFrame	readAvro(String path)	Read an Avro file as a DataFrame without registering it.
DataFrame	readAvro(String path, AvroReadOptions options)	Read an Avro file as a DataFrame with the supplied AvroReadOptions.
DataFrame	readCsv(String path)	Read a CSV file as a DataFrame without registering it.
DataFrame	readCsv(String path, CsvReadOptions options)	Read a CSV file as a DataFrame with the supplied CsvReadOptions.
DataFrame	readJson(String path)	Read a newline-delimited JSON file as a DataFrame without registering it.
DataFrame	readJson(String path, NdJsonReadOptions options)	Read a newline-delimited JSON file as a DataFrame with the supplied NdJsonReadOptions.
DataFrame	readParquet(String path)	Read a parquet file as a DataFrame without registering it.
DataFrame	readParquet(String path, ParquetReadOptions options)	Read a parquet file as a DataFrame with the supplied ParquetReadOptions.
void	registerArrow(String name, String path)	Register an Arrow IPC file (or directory of Arrow IPC files) as a table.
void	registerArrow(String name, String path, ArrowReadOptions options)	Register an Arrow IPC file (or directory of Arrow IPC files) as a table with the supplied ArrowReadOptions.
void	registerAvro(String name, String path)	Register an Avro file (or directory of Avro files) as a table.
void	registerAvro(String name, String path, AvroReadOptions options)	Register an Avro file (or directory of Avro files) as a table with the supplied AvroReadOptions.
void	registerCsv(String name, String path)
void	registerCsv(String name, String path, CsvReadOptions options)	Register a CSV file (or directory of CSV files) as a table with the supplied CsvReadOptions.
void	registerJson(String name, String path)
void	registerJson(String name, String path, NdJsonReadOptions options)	Register a newline-delimited JSON file (or directory of NDJSON files) as a table with the supplied NdJsonReadOptions.
void	registerParquet(String name, String path)
void	registerParquet(String name, String path, ParquetReadOptions options)	Register a parquet file as a table with the supplied ParquetReadOptions.
void	registerTable(String name, TableProvider provider)	Register a Java-implemented TableProvider under name.
void	registerUdf(ScalarUdf udf)	Register a Java-implemented scalar UDF.
DataFrame	sql(String query)	Parse and plan query, returning a lazy DataFrame.
org.apache.arrow.vector.types.pojo.Schema	tableSchema(String tableName)	Return the Arrow Schema of a registered table.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

SessionContext

public SessionContext()

Method Details

builder

public static SessionContextBuilder builder()

Start configuring a SessionContext.

sql

public DataFrame sql(String query)

Parse and plan query, returning a lazy DataFrame. The query is not executed until DataFrame.collect(org.apache.arrow.memory.BufferAllocator) is called.

fromProto

public DataFrame fromProto(byte[] planBytes)

Decode a DataFusion-Proto LogicalPlanNode and return a lazy DataFrame. The plan is not executed until DataFrame.collect(org.apache.arrow.memory.BufferAllocator) is called.

The bytes must be a serialized datafusion.LogicalPlanNode (see org.apache.datafusion.protobuf.LogicalPlanNode).

Throws:

RuntimeException - if the bytes are not a valid LogicalPlanNode or if logical planning fails.

tableSchema

public org.apache.arrow.vector.types.pojo.Schema tableSchema(String tableName)

Return the Arrow Schema of a registered table. Transferred via Arrow IPC; no BufferAllocator is required because a schema carries no buffer data.

Throws:

RuntimeException - if tableName is not registered in this context.

getOption

public String getOption(String key)

Read the current value of a datafusion.* config key. The key must be one DataFusion recognises (see SessionContextBuilder.setOption(String, String) for examples and the upstream configuration reference for the full list).

datafusion.runtime.* keys (memory limit, temp directory, cache sizes, etc) are not yet supported by this getter and will throw. Use the typed SessionContextBuilder.memoryLimit(long, double) and SessionContextBuilder.tempDirectory(String) setters at construction time instead. Round-trip support for the runtime subtree is tracked as a follow-up.

Returns:

the current value as a string, or null if the key is recognised but has no value set and no default.

Throws:

IllegalArgumentException - if key is null.

RuntimeException - if the key is not recognised by DataFusion or is in the datafusion.runtime.* subtree.

IllegalStateException - if this context is closed.

registerCsv

public void registerCsv(String name,
 String path)

registerCsv

public void registerCsv(String name,
 String path,
 CsvReadOptions options)

Register a CSV file (or directory of CSV files) as a table with the supplied CsvReadOptions.

Throws:

RuntimeException - if registration fails (path not found, schema inference error, etc.).

readCsv

public DataFrame readCsv(String path)

Read a CSV file as a DataFrame without registering it.

readCsv

public DataFrame readCsv(String path,
 CsvReadOptions options)

Read a CSV file as a DataFrame with the supplied CsvReadOptions.

Throws:

RuntimeException - if the read fails.

registerJson

public void registerJson(String name,
 String path)

registerJson

public void registerJson(String name,
 String path,
 NdJsonReadOptions options)

Register a newline-delimited JSON file (or directory of NDJSON files) as a table with the supplied NdJsonReadOptions.

Throws:

RuntimeException - if registration fails (path not found, schema inference error, etc.).

readJson

public DataFrame readJson(String path)

Read a newline-delimited JSON file as a DataFrame without registering it.

readJson

public DataFrame readJson(String path,
 NdJsonReadOptions options)

Read a newline-delimited JSON file as a DataFrame with the supplied NdJsonReadOptions.

Throws:

RuntimeException - if the read fails.

registerParquet

public void registerParquet(String name,
 String path)

registerParquet

public void registerParquet(String name,
 String path,
 ParquetReadOptions options)

Register a parquet file as a table with the supplied ParquetReadOptions.

Throws:

RuntimeException - if registration fails (path not found, schema mismatch, etc.).

readParquet

public DataFrame readParquet(String path)

Read a parquet file as a DataFrame without registering it.

readParquet

public DataFrame readParquet(String path,
 ParquetReadOptions options)

Read a parquet file as a DataFrame with the supplied ParquetReadOptions.

Throws:

RuntimeException - if the read fails.

registerArrow

public void registerArrow(String name,
 String path)

Register an Arrow IPC file (or directory of Arrow IPC files) as a table.

registerArrow

public void registerArrow(String name,
 String path,
 ArrowReadOptions options)

Register an Arrow IPC file (or directory of Arrow IPC files) as a table with the supplied ArrowReadOptions.

Throws:

IllegalArgumentException - if any of name, path, or options is null.

RuntimeException - if registration fails (path not found, schema mismatch, etc.).

readArrow

public DataFrame readArrow(String path)

Read an Arrow IPC file as a DataFrame without registering it.

readArrow

public DataFrame readArrow(String path,
 ArrowReadOptions options)

Read an Arrow IPC file as a DataFrame with the supplied ArrowReadOptions.

Throws:

IllegalArgumentException - if path or options is null.

RuntimeException - if the read fails.

registerAvro

public void registerAvro(String name,
 String path)

Register an Avro file (or directory of Avro files) as a table.

registerAvro

public void registerAvro(String name,
 String path,
 AvroReadOptions options)

Register an Avro file (or directory of Avro files) as a table with the supplied AvroReadOptions.

Throws:

IllegalArgumentException - if any of name, path, or options is null.

RuntimeException - if registration fails (path not found, schema mismatch, etc.).

readAvro

public DataFrame readAvro(String path)

Read an Avro file as a DataFrame without registering it.

readAvro

public DataFrame readAvro(String path,
 AvroReadOptions options)

Read an Avro file as a DataFrame with the supplied AvroReadOptions.

Throws:

IllegalArgumentException - if path or options is null.

RuntimeException - if the read fails.

registerUdf

public void registerUdf(ScalarUdf udf)

Register a Java-implemented scalar UDF. After registration, the function can be invoked by SQL via the UDF's name or referenced in DataFusion plans deserialised with fromProto(byte[]).

The UDF is registered with an exact signature: the runtime will reject calls whose argument types do not match the declared ScalarFunction.argFields() exactly.

Throws:

RuntimeException - if registration fails (e.g., name already registered with an incompatible signature, schema serialisation failure).

registerTable

public void registerTable(String name,
 TableProvider provider)

Register a Java-implemented TableProvider under name. SQL queries that reference name call back into provider to fetch batches.

TableProvider.schema() is called once here, on the calling thread, and cached on the native side. TableProvider.scan(org.apache.arrow.memory.BufferAllocator) is called once per query that touches the table, on a Tokio worker thread; it must return a fresh, independent ArrowReader on every call, with its buffers allocated from the BufferAllocator the framework supplies.

This is the Java counterpart to DataFusion's Rust SessionContext::register_table.

Throws:

IllegalArgumentException - if name or provider is null.

IllegalStateException - if provider.schema() returns null, or this context is closed.

RuntimeException - if native registration fails.

close

public void close()

Specified by:

close in interface AutoCloseable