BenchDiff

BenchDiff is a visual tool for spotting performance regressions and improvements in Google Benchmark traces.

Unlike the compare.py script bundled with Google Benchmark, BenchDiff provides:

• Clear visual output with severity-based coloring (minor/moderate/major)
• Per-kernel aggregation to easily analyze templated benchmarks
• CI mode with configurable thresholds and exit codes

Install

Requires Python 3.10+

pip install benchdiff

Quick Start

# Basic comparison
benchdiff --ref baseline.json --cur current.json

# Aggregated view (per-kernel summary)
benchdiff --ref baseline.json --cur current.json --aggregate-only

# CI mode (exit 4 on major regression)
benchdiff --ref baseline.json --cur current.json --ci --ci-fail-on major

Try with included sample data:

benchdiff --ref demo/baseline_O2.json --cur demo/current_O3_native.json

Documentation

See the docs folder for detailed documentation:

• Usage Guide — CLI options and examples
• Development — Architecture, API, contributing

License

BSD 3-Clause