Benchmarking#

Current TPC-H SF1000 results for Ballista, compared against vanilla Spark 4.1.3 and Spark 4.1.3 with Apache DataFusion Comet 1.1.0-rc2, running on the same cluster shape. A Trino 483 column is included for reference; it is not an apples-to-apples comparison (see Trino configuration).

Versions under test#

Engine

Version

Ballista

c76278cd (main, 2026-10-07), Cargo pkg 55.0.0, DataFusion 55.1.0

Spark

4.1.3 (apache/spark:4.1.3 image, vanilla, no acceleration plugin)

Comet

Apache DataFusion Comet 1.1.0-rc2 (992c806a) on the same Spark 4.1.3

Trino

483 (trinodb/trino:483 image, Hive connector)

Results#

Times in seconds; lower is better. Ballista, Spark, and Spark + Comet: mean of 2 iterations, including the cold first iteration. Trino: mean of 3 iterations after an untimed warm-up run (see Trino configuration).

Query

Ballista (s)

Spark 4.1.3 (s)

Spark 4.1.3 + Comet 1.1.0-rc2 (s)

Trino 483 (s)

1

14.07

71.45

10.66

5.32

2

25.81

37.98

21.66

15.34

3

17.22

30.61

16.05

13.47

4

15.31

23.86

8.11

8.01

5

46.88

57.39

38.32

25.98

6

1.59

1.73

0.99

0.84

7

37.26

25.34

16.60

16.19

8

20.13

58.17

46.04

30.93

9

35.94

74.90

55.78

41.71

10

43.36

35.31

21.31

20.41

11

15.01

31.66

14.79

5.73

12

10.49

13.24

5.33

4.40

13

11.05

22.73

11.31

10.56

14

5.54

7.25

2.50

1.66

15

10.58

22.45

11.85

8.30

16

12.78

24.34

8.94

6.98

17

20.74

81.36

25.14

24.76

18

45.63

134.11

46.42

66.00

19

14.59

13.27

9.07

5.93

20

14.46

16.08

6.57

8.57

21

75.74

98.04

68.43

83.22

22

8.25

16.90

9.22

7.29

Total

502.43

898.17

455.09

411.62

Reproducing#

The Ballista numbers come from the tpch runner in this repository. The Spark, Spark + Comet, and Trino numbers were collected with a benchmark harness that is not in this repository yet. We are working on adding scripts so that anyone can reproduce all of these results. See the per-engine pages below for the commands and settings.

Details#

How the cluster was set up and how each engine was configured:

Recording a new result set#

  • Pin the exact commit the numbers came from, not a branch name.

  • Replace the version, environment, config, and results tables together — a row that mixes numbers from different commits silently misattributes a regression.

  • Prefer a single continuous suite run: a long-lived executor deep into a suite is not in the same state as a freshly started one.

  • Report FAIL for a query that ran but did not produce an answer, and OOM when the failure is a known memory exhaustion. See the tracker for open issues found by benchmarking: #1359, #2025, #2063.