Ladybird Web Benchmarks

This repository contains a set of web benchmarks used to measure the performance of the Ladybird browser engine.

Installation

To install the python packages required by the benchmark runner you must install the packages listed in requirements.txt. You can do this using pip:

pip install -r requirements.txt

Running Benchmarks

Before running the benchmarks you must first build Ladybird using the build instructions for Ladybird. For best results, it is recommended to build Ladybird with the Distribution build preset, like so:

BUILD_PRESET=Distribution ./Meta/ladybird.py build ladybird

After Ladybird is built, benchmarks are run using the run.py script. You must provide the path to the Ladybird executable using the --executable argument. For example:

./run.py --executable "${LADYBIRD_SOURCE_DIR}/Build/distribution/bin/ladybird" --output results.json  

Use --benchmarks to select one or more benchmarks. The imported WebKit performance suites are available as WebKitBindings, WebKitCSS, WebKitDOM, WebKitParser, and WebKitSVG:

./run.py --executable "${LADYBIRD_SOURCE_DIR}/Build/distribution/bin/ladybird" \
    --benchmarks WebKitDOM,WebKitCSS --output results.json

These suites were imported from WebKit revision 2e5d042491f325b2df778dbba97c48d66bf2d395. They run one sample per test by default; pass --iterations to choose another sample count. Runs-per-second tests use fixed batch sizes and report the elapsed time for that workload, so faster code also shortens the benchmark run. Each suite's Skipped file records tests excluded by WebKit or by this runner.

MicroWeb is a home-grown suite of ~230 fine-grained time-based microbenchmarks covering the primitives Speedometer3 exercises (DOM API, render pipeline, layout modes, style invalidation, text shaping, JS, canvas, SVG, parsing, shadow DOM, events, editing, observers, timers, URL/history, storage). See benchmarks/MicroWeb/README.md for the comparison workflow against Chromium.

To run any benchmark under Chromium for comparison, pass --browser chromium (and typically --jitless, which disables the V8 JIT for an engine-vs-engine comparison):

./run.py --executable /snap/bin/chromium --browser chromium --jitless \
    --benchmarks MicroWeb -o chromium.json

ratios.py then ranks every test by the subject/reference time multiplier, worst first:

./ratios.py --subject ladybird.json --reference chromium.json

Comparing Results

After running benchmarks and saving the results as a JSON file, you can compare the results using the compare.py script. For example:

./compare.py -o old.json -n new.json