[turbopack] Optimize compaction cpu usage#91468
Merged
lukesandberg merged 10 commits intocanaryfrom Mar 20, 2026
Merged
Conversation
Contributor
Tests Passed |
Contributor
Stats from current PR✅ No significant changes detected📊 All Metrics📖 Metrics GlossaryDev Server Metrics:
Build Metrics:
Change Thresholds:
⚡ Dev Server
📦 Dev Server (Webpack) (Legacy)📦 Dev Server (Webpack)
⚡ Production Builds
📦 Production Builds (Webpack) (Legacy)📦 Production Builds (Webpack)
📦 Bundle SizesBundle Sizes⚡ TurbopackClient Main Bundles
Server Middleware
Build DetailsBuild Manifests
📦 WebpackClient Main Bundles
Polyfills
Pages
Server Edge SSR
Middleware
Build DetailsBuild Manifests
Build Cache
🔄 Shared (bundler-independent)Runtimes
📎 Tarball URL |
Merging this PR will not alter performance
Comparing Footnotes
|
a5f38eb to
3600f32
Compare
7610827 to
4a99afc
Compare
3600f32 to
4a942b8
Compare
Member
|
bgw
approved these changes
Mar 17, 2026
9800c36 to
56100a3
Compare
lukesandberg
commented
Mar 18, 2026
f25dedc to
58b70cb
Compare
lukesandberg
commented
Mar 18, 2026
lukesandberg
commented
Mar 18, 2026
lukesandberg
commented
Mar 18, 2026
58b70cb to
c8f0fba
Compare
Contributor
Author
Merge activity
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

Summary
Optimizes the turbo-persistence compaction and iteration paths with several targeted improvements:
Iterator optimizations
Flatten index block iteration — The iterator previously used a
Vec<CurrentIndexBlock>stack, but SST files have exactly one index level. Inline the index block fields (index_entries,index_block_count,index_pos) directly intoStaticSortedFileIter, eliminating the stack allocation andOptionoverhead.Non-optional
CurrentKeyBlock— Parse the first key block duringtry_into_iter()construction socurrent_key_blockis always populated, removing theOption<CurrentKeyBlock>wrapper and itstake()/Some()ceremony in the hot loop.Replace
ReadBytesExtwith direct byte indexing — Inhandle_key_match,parse_key_block, andnext_internal, replaceval.read_u16::<BE>()etc. withu16::from_be_bytes(val[0..2].try_into().unwrap()). This eliminates th mutable slice pointer advancement.Extract
read_offset_entryhelper — Read type + offset from the key block offset table in a singleu32load + shift, replacing two separateReadBytesExtcalls.Refcounting optimization
RcBytes— Thread-local byte slice type usingRcinstead ofArc, eliminating atomic refcount overhead during single-threaded SST iteration. The iteration path (StaticSortedFileIter) now producesRcBytesslices backed by anRc<Mmap>, so per-entry clone/drop operations are plain integer increments rather than atomic operations.Merge iterator simplification
MergeIter::next— Replaced the straightforwardspop/pushpattern withPeekMut-based replace-top pattern, which means we only need to adjust the heap once per iteration instead of twice.Benchmark results
Compaction (
key_8/value_4/entries_16.00Mi/commits_128)Read path (
static_sorted_file_lookup/entries_1000.00Ki)No read regression — the branch is neutral to slightly faster:
Test plan
cargo test -p turbo-persistence— 60/60 tests passing