experiment: try LTO for Linux native builds (ELF only)#72
Conversation
Adds .github/workflows/benchmark-lto.yml (workflow_dispatch only - JMH runs are too slow for every push/PR, and this is throughput exploration, not a correctness gate). For each of linux-x86_64, linux-aarch64, osx-aarch64, windows-x86_64: checks out main and an LTO variant branch (default experiment/lto-linux, overridable via the lto_ref input) side by side, builds+runs the CompressBenchmark/DecompressBenchmark suite (quick smoke settings: 1 fork, 2 warmup, 5 measurement, size=65536) against each, and writes a comparison table to the job summary via the new .github/scripts/compare-benchmarks.py. This turns the "-flto is Linux-only, no-op elsewhere" claim from #70/ #72 into CI evidence per platform instead of a manual assertion - linux-x86_64/linux-aarch64 should show a real delta, osx-aarch64/ windows-x86_64 should show ~0% (LTO_FLAG stays empty for those classifiers in scripts/build-zstd.sh). Verified locally: - actionlint clean - compare-benchmarks.py tested against real JMH JSON output; found and fixed a real bug along the way - scoreError comes through as the JSON string "NaN" (not a number) when JMH can't compute a confidence interval, e.g. with few iterations
Add -flto for linux-* classifiers only. zig's Mach-O backend is a
self-hosted linker with no LTO support ("error: LTO requires using
LLD" even at the -c compile stage) — confirmed no workaround exists
locally (no real ld64.lld available, zig cc doesn't expose a linker
toggle). ELF is different: zig 0.16 still links it through the
bundled real lld by default, and -flto compiles and links cleanly
when cross-compiled to x86_64-linux-gnu/aarch64-linux-gnu from this
host, with the full public API surface (verified via
llvm-readelf --dyn-syms) intact after LTO.
What isn't verified locally: whether the LTO-linked library is
actually correct at runtime — this Mac can't execute Linux ELF
binaries. zstd/pom.xml and integration-tests/pom.xml activate their
native-classifier profile by host OS/arch, so ci.yml's ubuntu-latest
job will natively build and test-run exactly the linux-x86_64
classifier this touches. Marking this experimental pending that
signal, rather than folding it into #71 (asm/arch/hardening), which
only contains changes verified end-to-end on this host.
osx-*/windows-* classifiers are untouched (LTO_FLAG stays empty);
confirmed unaffected locally (rebuilt + zstd/integration-tests suite
passes on osx-aarch64, all six classifiers still cross-compile).
Ref #70
e50d7ed to
459f6c7
Compare
Benchmark results: LTO doesn't help, mildly hurts on x86_64Ran the new
Bottom line: LTO doesn't move the needle positively anywhere, and measurably hurts x86_64 compression throughput. This is a single quick JMH run (1 fork / 2 warmup / 5 measurement — directional per ADR 0012, not publication-grade), but the direction is consistent and the control comparisons rule out noise as the explanation on Linux. Leaving this open rather than merging - the data doesn't support it as-is. Possible next steps if we want to revisit: a heavier/more-iterations run to confirm, or investigating why LTO regresses zstd's compress path here (e.g. it may be pessimizing branch layout or inlining decisions the hand-tuned zstd C code already relies on) before trying again. |
|
Closing without merging. A second measurement (main vs this branch, run 29634524084) replicates the original finding and makes it look worse, not better:
Two independent runs now agree LTO consistently regresses compress throughput on x86_64, with no reliable decompress upside to offset it. Combined with the macOS blocker (zig's Mach-O linker has no LTO support at all), this isn't worth pursuing further. Full writeup and first measurement on #70. |
1 participant
Summary
Experimental follow-up to #70 / #71: enable
-fltofor thelinux-x86_64/linux-aarch64classifiers only.error: LTO requires using LLD, reproducible even at the-ccompile stage, no workaround found — no realld64.lldlocally,zig ccexposes no linker-selection flag that helps).lldby default (the self-hosted ELF linker is early/opt-in and, per the 0.16 devlog, can't even link external-library code yet — not usable for us regardless).-fltocompiles and links cleanly when cross-compiled locally tox86_64-linux-gnu/aarch64-linux-gnu, and the full public API surface survives (checked withllvm-readelf --dyn-syms—ZSTD_compress,ZSTD_decompress,ZSTD_createCCtx,ZDICT_trainFromBuffer, etc. all still exported, ~198 public symbols).Why this is a separate PR from #71 and marked experimental: everything in #71 was verified end-to-end on this (macOS) host, including running the actual test suite against the built library. This one I can't fully verify locally — this Mac can't execute Linux ELF binaries, so linking without error is as far as local verification goes.
zstd/pom.xml/integration-tests/pom.xmlactivate their native-classifier Maven profile by host OS/arch, soci.yml'subuntu-latestjob will natively build and test-run exactly thelinux-x86_64classifier this PR touches — that's the real correctness signal I'm after here. If CI is green, this is solid; if not, we'll know exactly where it breaks.osx-*/windows-*are untouched (confirmed: rebuilt osx-aarch64, full zstd/integration-tests suite still passes; all six classifiers still cross-compile cleanly).Test plan
linux-x86_64/linux-aarch64locally with-flto, both.ocompile and.solink succeedllvm-readelf --dyn-symsconfirms full public API surface survives LTO./mvnw -q -pl zstd,integration-tests -am testand./mvnw -q validate(checkstyle) both passlinux-x86_64/linux-aarch64— this is the actual runtime-correctness check for this PR, not just a formality🤖 Generated with Claude Code