Package org.apache.datafusion

Class ObjectStoreOptions

java.lang.Object
org.apache.datafusion.ObjectStoreOptions
Direct Known Subclasses:
ObjectStoreOptions.Gcs, ObjectStoreOptions.Http, ObjectStoreOptions.S3
public abstract sealed class ObjectStoreOptions extends Object permits ObjectStoreOptions.S3, ObjectStoreOptions.Gcs, ObjectStoreOptions.Http
Description of an object_store::ObjectStore backend to register on a SessionContext's RuntimeEnv. Pass instances to SessionContextBuilder.registerObjectStore(ObjectStoreOptions).

Build instances via the per-backend factories: s3(), gcs(), http(String). Each factory returns a builder whose build() produces an immutable ObjectStoreOptions you can register.

For the cloud backends (S3 / GCS), the standard SDK environment variables are read as the default floor (e.g. AWS_ACCESS_KEY_ID, AWS_DEFAULT_REGION, ECS / web-identity vars; GOOGLE_APPLICATION_CREDENTIALS). Explicit setters on the per-backend builder always override those env values, so a JVM running on a credentialed host (IAM role, ECS task, GKE workload identity) can register an empty-credentials store and let the SDK's default chain do its job.

The URL DataFusion uses to look up the store is either an explicit url(...) on the builder or, if unset, derived from the typed fields:

  • S3 → s3://<bucket>
  • GCS → gs://<bucket>
  • HTTP → no scheme-default; http(String) requires the URL up front. The URL must be a host root (https://example.com/ or https://example.com); paths cause lookup-vs-base-URL collisions in DataFusion's registry. Register one HTTP store per scheme+host and let SQL paths carry the rest.