Comet Benchmarking Guide

To track progress on performance, we regularly run benchmarks derived from TPC-H and TPC-DS.

The benchmarking scripts are contained here.

Data generation scripts are available in the DataFusion Benchmarks GitHub repository.

Current Benchmark Results

• Benchmarks derived from TPC-DS: Comet 1.1.0 and 1.2.0 vs. Spark 4.2.0 and 4.1.3 on Parquet tables at SF 1000

Benchmarking Guides

Available benchmarking guides:

• Benchmarking on macOS

• Benchmarking on AWS EC2

• Micro Benchmarking on AWS EC2

• TPC-DS Benchmarking with spark-sql-perf

We also have many micro benchmarks that can be run from an IDE located here. These can also be run as a suite on a dedicated machine, see Micro Benchmarking on AWS EC2. Published results are in benchmarks/results/micro.