Profiling Cookbook¶
The section contains examples how to perform CPU profiling for Apache DataFusion on different operating systems.
MacOS¶
Building a flamegraph¶
Video: how to CPU profile DataFusion with a Flamegraph
A flamegraph is a visual representation of which functions are being run You can create flamegraphs in many ways; The instructions below are for cargo-flamegraph which results in images such as this:
To create a flamegraph, you need to install the flamegraph tool:
cargo install flamegraph
Then you can run the flamegraph tool with the -- separator to pass arguments
to the binary you want to profile.
Example: Flamegraph for datafusion-cli executing q28.sql. Note this
must be run as root on Mac OSx to access DTrace.
sudo flamegraph -- datafusion-cli -f q28.sq
You can also invoke the flamegraph tool with cargo to profile a specific test or benchmark.
Example: Flamegraph for a specific test:
CARGO_PROFILE_RELEASE_DEBUG=true cargo flamegraph --root --unit-test datafusion -- dataframe::tests::test_array_agg
Example: Flamegraph for a benchmark
CARGO_PROFILE_RELEASE_DEBUG=true cargo flamegraph --root --bench sql_planner -- --bench