Skip to main content
Ctrl+K
Apache DataFusion in Python Apache DataFusion in Python
  • User Guide
  • Extension Guide
  • Contributor Guide
  • API Reference
  • Links
  • GitHub
  • Rust API docs (docs.rs)
  • User Guide
  • Extension Guide
  • Contributor Guide
  • API Reference
  • Links
  • GitHub
  • Rust API docs (docs.rs)

Section Navigation

  • User Guide
    • Introduction
    • Concepts
    • Data Sources
    • DataFrames
      • DataFrame Rendering
      • Execution Metrics
    • Common Operations
      • Registering Views
      • Basic Operations
      • Column Selections
      • Expressions
      • Joins
      • Functions
      • Spark-Compatible Functions
      • Aggregation
      • Window Functions
      • User-Defined Functions
    • IO
      • Arrow
      • Avro
      • CSV
      • JSON
      • Parquet
      • Custom Table Provider
    • Configuration
    • Using extension libraries
    • Distributing work
      • Shipping expressions to workers
      • Distributed query engines
    • SQL
    • Upgrade Guides
    • Using AI Coding Assistants
  • Extension Guide
    • Why FFI
    • The capsule protocol
    • Providers and catalogs
    • Functions and table functions
    • Extension codecs
    • Extension bundles
    • Query planners
    • Optimizer rules and configuration
    • Sessions, handles, and lifetimes
    • Extension author checklist
  • Contributor Guide
    • Introduction
    • PyO3 binding guidelines
    • FFI framing internals
  • API Reference
    • datafusion
      • datafusion.catalog
      • datafusion.context
      • datafusion.dataframe
      • datafusion.dataframe_formatter
      • datafusion.expr
      • datafusion.extensions
      • datafusion.functions
        • datafusion.functions.spark
      • datafusion.input
        • datafusion.input.base
        • datafusion.input.location
      • datafusion.io
      • datafusion.ipc
      • datafusion.object_store
      • datafusion.options
      • datafusion.plan
      • datafusion.record_batch
      • datafusion.substrait
      • datafusion.unparser
      • datafusion.user_defined
  • Links
    • GitHub and Issue Tracker
    • Rust API Docs
    • Code of Conduct
    • Examples
  • Contributor Guide

Contributor Guide#

Guides for contributors to the DataFusion in Python project — people changing this repository itself.

If you are writing a separate library that plugs into datafusion-python, you want the Extension Guide instead. If you are using one, see Using extension libraries.

  • Introduction
    • How to develop
    • Running & Installing pre-commit hooks
    • Guidelines for Separating Python and Rust Code
    • Update Dependencies
    • Improving Build Speed
  • PyO3 binding guidelines
    • Class mutability
  • FFI framing internals
    • One session, one Arc<SessionContext>
    • Why planner codec rebinding is one level deep
    • Why with_extensions commits last
    • Two argument kinds for one convention

previous

Extension author checklist

next

Introduction

Apache Arrow DataFusion, Arrow DataFusion, Apache, the Apache feather logo, and the Apache Arrow DataFusion project logo

are either registered trademarks or trademarks of The Apache Software Foundation in the United States and other countries.