Comet Contributor Guide# Contributor Guide Getting Started Finding issues to work on Reporting issues Asking for Help Regular public meetings Comet Plugin Architecture Overview Plugin Components Comet SQL Plugin CometSparkSessionExtensions Query Execution Shuffle Arrow FFI Overview Arrow FFI Basics JVM → Native Data Flow (ScanExec) Native → JVM Data Flow (CometExecIterator) Memory Ownership Rules Further Reading Parquet Scans S3 Support Development Guide Project Layout Development Setup Build & Test Development Environment Plan Stability Testing Benchmark Debugging Submitting a Pull Request How to format .md document Debugging Guide Debugging for Advanced Developers Verbose debug Benchmarking Guide Current Benchmark Results Benchmarking Guides Adding a New Operator Overview Implementing a Native Operator Implementing a Sink Operator Implementing a JVM Operator Common Patterns and Helpers Debugging Tips Adding a New Expression Finding an Expression to Add Implementing the Expression Shimming to Support Different Spark Versions Resources Tracing Definition of Labels Profiling Native Code Running micro benchmarks with cargo bench Profiling with cargo-flamegraph Spark SQL Tests 1. Install Comet 2. Clone Spark and Apply Diff 3. Run Spark SQL Tests Creating a diff file for a new Spark version Generating The Diff File Running Tests in CI Roadmap Major Initiatives Ongoing Improvements Github and Issue Tracker
