Research: seven-component aggregation performance stack

[latifkasuli]

Summary

This draft preserves an evidence-backed aggregation performance research stack and integrates upstream main at e9cd16d49ef33909d9732778451ec73fbbedfd4a.

It is a research handoff, not a merge-ready patch. Replay selectors and the measurement seam remain so the retained campaigns are reproducible. The mechanisms should be split into smaller review units after the target architecture and acceptance budget are agreed.

Source and evidence boundary

• The six-component all-off/on campaign binds to b61a0ee64a9e5f41c368f937c56cdbb74fd3908b on the older 84fbd3e freeze.
• The incremental L0/seven-component campaign binds to 256928f9127c812749e40b08b4cf9744185c6b61 on that same freeze.
• The fresh current-source two-kernel campaign binds to performance-bearing commit a5477b369ee44ef7aea91b0799f7b920b349632f on integrated upstream e9cd16d.
• Current PR head e8e873f47f4f03743f1e1bbc95e26026063016e8 adds only review hardening, exactness coverage and evidence-disclosure corrections after the measured commit. It has no replacement performance campaign.

The earlier six-component and L0 percentages are not relabeled as current-main results.

Earlier seven-component result

On the older measured freeze:

• Six components versus all off at N2: outer proving −30.91%, process wall −20.72%, peak physical memory −22.98%.
• Six components versus all off at N8: outer proving −31.87%, process wall −26.85%, peak physical memory −32.44%.
• Incremental L0 NTT at N2: L0 induction −78.02%, PCS opening −25.66%, outer proving −7.07%, process wall −3.51%, peak physical memory −2.48%.
• Incremental L0 NTT at N8: L0 induction −74.10%, PCS opening −24.32%, outer proving −6.51%, process wall −4.46%, peak physical memory −3.27%.
• All 142 retained proofs were byte-identical within topology and passed the unchanged inspection path.

After seven components, the N2 phase cluster was Bus 0.687050 s, PCS opening 0.527055 s, constraints 0.518037 s and Flock reduction 0.336672 s. This moves the previous PCS bottleneck; it does not prove that aggregation meets a production budget.

Current-source two-kernel result

Two default-off, fail-closed experiments were measured on the integrated stack:

1. LEANVM_CONSTRAINT_NODE_SKIP=1 derives one Boolean constraints-sumcheck endpoint from the running claim, including the exact zeta == 1 branch.
2. FLOCK_PACKED_128_PARALLEL=1 serializes the three live packed witnesses concurrently into disjoint, completely initialized outputs.

The canonical N2 campaign used one AMD EPYC 9354 NUMA domain, CPUs 8–15, four pilots and 32 measured fresh processes in a repeated 2×2 Williams-square design.

Effect	Paired-block median	Favorable blocks
Node skip on constraints	−62.897 ms	8/8
Parallel path on packed copy	−118.250 ms	8/8
Parallel path on Flock reduction	−124.799 ms	8/8
Combined versus control on outer proving	−189.041 ms (−6.31%)	8/8
Combined versus control on process wall	−175.106 ms (−3.02%)	8/8
Combined versus control on peak physical memory	−3.781 MB	5/8

All 36 proofs were byte-identical at 230,804 bytes, SHA-256 c05561327b52c3a11466511dc4ccde942d89086f4541b13eb9d27ae1cf0d3e79, and passed the unchanged inspection path. The 548-entry campaign manifest has SHA-256 d7f9bedf61556cd6b0d17f052d70d4e791dfb77047bca45e054ac47228d3f03a.

Methodology disclosure

The single-NUMA repeat was selected after the valid split-NUMA _02 campaign. In _02, packed copy and Flock were favorable in 8/8 blocks, constraints in 6/8, and combined outer and wall effects in only 4/8; effective CPU service varied from 2.136 to 2.885 cores.

The analysis decision record was not a prospective preregistration. It was authored after 16 measured processes had completed and copied to the host after 18 of 32 had completed. The exposed console observation contained admission lines rather than outcomes, but non-observation of already-written result files is not independently provable.

Under that record, the mechanism and direction gates passed, but the system-materiality gate required at least −350 ms of outer-prove improvement and failed at −189.041 ms. The direct mechanism measurements are exact and repeatedly favorable. The system deltas are descriptive paired measurements, not a confirmatory end-to-end or production-acceptance claim.

Review hardening

• write_packed_128 now requires [MaybeUninit<u8>; 16], making complete initialization a type-level contract with a release-mode boundary check rather than a debug-only length assertion.
• The constraints transcript-equivalence test now executes the parallel two-node reducer at tau = 12, including the exceptional zeta == 1 recovery path.
• The observed 13.59% constraints reduction from removing one of three evaluations suggests, under an equal-evaluation-cost assumption, that roughly 59% of the phase remains traversal/materialization. This is a calibration inference, not direct instrumentation.

Validation

On exact measured commit a5477b3, the sealed Linux receipt records cargo testall: 293 passed, 0 failed and 10 ignored, including doctests. Clippy, formatting and documentation also passed.

On review head e8e873f, both complete local release suites, Clippy, formatting and documentation passed. This is regression/exactness evidence, not a sealed replacement performance campaign. GitHub reported no CI checks when this body was prepared.

The full external evidence pack contains raw run order, commands, environment, memory samples, phase records, serialized proofs, inspection results, source bundles and checksum manifests and can be transferred separately. RESEARCH_HANDOFF.md contains the detailed source identities and fast-upstream procedure.

Feedback requested

1. What N2 latency, peak-memory, proof-size and concurrency budgets should define acceptance?
2. Is current main the intended architecture for splitting and productionizing these changes?
3. Which mechanism should be reviewed first: direct fold-6, the allocation/capacity group, AVX-512 Bus MLE, the narrow L0 induction policy, the constraints node skip, or packed serialization?