Comet Contributor Guide

Contributor Guide

• Getting Started
  • Finding issues to work on
  • Reporting issues
  • Asking for Help
  • Regular public meetings
• Comet Plugin Architecture
  • Overview
• Plugin Components
  • Comet SQL Plugin
  • CometSparkSessionExtensions
  • Query Execution
  • Shuffle
  • Arrow FFI
  • End to End Flow
• Development Guide
  • Project Layout
  • Development Setup
  • Build & Test
  • Development Environment
  • Plan Stability Testing
  • Benchmark
  • Debugging
  • Submitting a Pull Request
• Debugging Guide
  • Debugging for Advanced Developers
  • Verbose debug
• Benchmarking Guide
  • Current Benchmark Results
  • Benchmarking Guides
• Adding a New Expression
  • Finding an Expression to Add
  • Implementing the Expression
  • Shimming to Support Different Spark Versions
  • Resources
• Tracing
  • Definition of Labels
• Profiling Native Code
  • Running micro benchmarks with cargo bench
  • Profiling with cargo-flamegraph
• Spark SQL Tests
  • 1. Install Comet
  • 2. Clone Spark and Apply Diff
  • 3. Run Spark SQL Tests
  • Creating a diff file for a new Spark version
  • Generating The Diff File
  • Running Tests in CI
• Roadmap
  • Major Initiatives
  • Ongoing Improvements
• Github and Issue Tracker