Comet Benchmarking Guide

To track progress on performance, we regularly run benchmarks derived from TPC-H and TPC-DS.

The benchmarking scripts are contained here.

Data generation scripts are available in the DataFusion Benchmarks GitHub repository.

Current Benchmark Results

• Benchmarks derived from TPC-DS

Benchmarking Guides

Available benchmarking guides:

• Benchmarking on macOS

• Benchmarking on AWS EC2

• Micro Benchmarking on AWS EC2

• TPC-DS Benchmarking with spark-sql-perf

We also have many micro benchmarks that can be run from an IDE located here. These can also be run as a suite on a dedicated machine, see Micro Benchmarking on AWS EC2. Published results are in benchmarks/results/micro.