Package org.apache.datafusion

Class SessionContextBuilder

java.lang.Object
org.apache.datafusion.SessionContextBuilder
public final class SessionContextBuilder extends Object
Builder for a configured SessionContext. Each setter is optional; unset fields leave the DataFusion default in place.

  • Method Details

    • batchSize

      public SessionContextBuilder batchSize(int batchSize)

    • targetPartitions

      public SessionContextBuilder targetPartitions(int targetPartitions)

    • collectStatistics

      public SessionContextBuilder collectStatistics(boolean collectStatistics)

    • informationSchema

      public SessionContextBuilder informationSchema(boolean informationSchema)

    • memoryLimit

      public SessionContextBuilder memoryLimit(long maxMemoryBytes, double fraction)
      Cap the memory pool at maxMemoryBytes, reserving fraction of it for queries.

    • tempDirectory

      public SessionContextBuilder tempDirectory(String path)
      Directory the DiskManager uses for spill files.

    • disableSpill

      public SessionContextBuilder disableSpill()
      Disable on-disk spill entirely. Queries that need spill fail with a ResourcesExhaustedException rather than going to disk; useful for memory-only execution profiles or environments without writable disk.

      Mutually exclusive with tempDirectory(String) — the combination throws at build() time. maxTempDirectorySize(long) is allowed alongside this setter but is a no-op (no directory to cap).

    • maxTempDirectorySize

      public SessionContextBuilder maxTempDirectorySize(long bytes)
      Cap the cumulative bytes used by spill files under tempDirectory(String). Mirrors upstream RuntimeEnvBuilder::with_max_temp_directory_size 1:1. Once exceeded, queries that need more spill space fail with a ResourcesExhaustedException. Combinable with disableSpill() but a no-op there.

      0 is accepted — upstream documents zero as legal and equivalent to "no spill allowed". Negative values are rejected.

      Throws:
      IllegalArgumentException - if bytes is negative.

    • setOption

      public SessionContextBuilder setOption(String key, String value)
      Set an arbitrary datafusion.* config option by string key. Mirrors DataFusion's ConfigOptions::set(key, value) API — see the DataFusion configuration reference for the full set of keys. Entries set this way are applied after the typed setters on this builder, so an explicit setOption call overrides a typed setter for the same knob.

      datafusion.runtime.* keys (memory limit, temp directory, cache sizes, etc) are not yet supported by this setter and will throw at build(). Use the typed memoryLimit(long, double) and tempDirectory(String) setters instead. Round-trip support for the runtime subtree is tracked as a follow-up.

      Unknown keys or unparseable values are not validated here; they surface as a RuntimeException from build() carrying DataFusion's error message.

      Throws:
      IllegalArgumentException - if key or value is null.

    • setOptions

      public SessionContextBuilder setOptions(LinkedHashMap<String,String> entries)
      Apply every entry of entries via setOption(String, String), in LinkedHashMap insertion order. Use this overload when you need the strict last-write-wins ordering guarantee for overlapping side-effect keys (e.g. datafusion.optimizer.enable_dynamic_filter_pushdown rewrites the per-operator enable_*_dynamic_filter_pushdown flags, so a per-operator override must come after the umbrella).
      Throws:
      IllegalArgumentException - if any key or value in entries is null.

    • setOptions

      public SessionContextBuilder setOptions(Map<String,String> entries)
      Apply every entry of entries via setOption(String, String). Iterates in whatever order the supplied Map produces, which for HashMap or Properties is unspecified.

      This is the right overload for the common case where the caller's keys don't overlap with any upstream setter's side effects. If you do need order — see the setOptions(LinkedHashMap) overload, which the compiler will resolve to automatically when you pass a LinkedHashMap.

      Throws:
      IllegalArgumentException - if any key or value in entries is null.

    • cacheManager

      public SessionContextBuilder cacheManager(CacheManagerOptions options)
      Configure DataFusion's built-in CacheManager for the new context. Build the CacheManagerOptions via CacheManagerOptions.builder(); each cache slot is independent, so leaving a setter unset keeps the upstream default in place.

      Calling this setter twice replaces the previous configuration — there is no incremental merge between calls. If you need a different cache configuration, build a new CacheManagerOptions from scratch.

      Throws:
      IllegalArgumentException - if options is null.

    • withSparkFunctions

      public SessionContextBuilder withSparkFunctions()
      Register Apache Spark-compatible functions and expression planners on the new context, using the datafusion-spark crate. Once enabled, Spark-compatible functions (e.g. crc32) are callable from SQL and override any DataFusion built-in of the same name.

      Requires the native library to be built with the spark Cargo feature, which is enabled in the default build. If it is not, build() throws a RuntimeException explaining the feature is missing.

    • registerObjectStore

      public SessionContextBuilder registerObjectStore(ObjectStoreOptions options)
      Register an object_store::ObjectStore backend on the new context's RuntimeEnv. Build ObjectStoreOptions via the per-backend factories (ObjectStoreOptions.s3(), ObjectStoreOptions.gcs(), ObjectStoreOptions.http(String)). The store is reachable inside the resulting SessionContext by URL — e.g. once an S3 store is registered for my-bucket, ctx.registerParquet("orders", "s3://my-bucket/orders/") will resolve through it.

      Multiple registrations are applied in the order added; if two registrations resolve to the same URL, the later one wins (matching upstream RuntimeEnv::register_object_store).

      If the underlying object_store cloud-backend Cargo feature is not built into the native library, build() surfaces a clear error rather than silently dropping the registration. The default make build enables all three backends (S3 / GCS / HTTP).

      Throws:
      IllegalArgumentException - if options is null.

    • build

      public SessionContext build()
      Construct a SessionContext with the configured options.
      Throws:
      IllegalStateException - if disableSpill() was called alongside tempDirectory(String) — the combination is contradictory.
      RuntimeException - if the native side fails to construct the context.