Skip to content

fix(buffer): reject ZSTs and address allocator review comments - #9807

Merged
gatesn merged 6 commits into
developfrom
ngates/buffer-review-minimal
Sep 8, 2026
Merged

fix(buffer): reject ZSTs and address allocator review comments#9807
gatesn merged 6 commits into
developfrom
ngates/buffer-review-minimal

Conversation

@gatesn

@gatesn gatesn commented Sep 8, 2026

Copy link
Copy Markdown
Contributor

Reject zero-sized buffer elements at construction with compile-time assertions. ZST support was previously inconsistent; rejecting these types prevents division-by-zero and pointer-distance failures without adding iterator or growth special cases. Empty buffers of non-ZST types remain supported. Six compile-fail doctests cover the independent construction paths.

Remove the generic raw-owner constructor. Typed Vec adoption takes its pointer and element count directly from the Vec and retains that Vec for destruction. External byte owners continue to use Bytes::from_owner at the call site, without adding a buffer API.

Address the allocator-stack review comments with field documentation, an explanation of the optional allocator, and tests for owner lifetime, allocator retention, empty/shared ownership, sliced-buffer growth, and alignment. The allocator representation, buffer layout, alignment policy, and allocation benchmarks are retained. Both iterator-extension implementations and the owned iterator match the merged baseline. This is a focused alternative to #9745.

Validation:

  • cargo nextest run -p vortex-buffer --all-features: 874 tests passed.
  • cargo test --doc -p vortex-buffer --all-features: 8 ordinary and 6 compile-fail doctests passed.
  • cargo clippy -p vortex-buffer --all-targets --all-features: passed.
  • Nightly formatting and git diff --check: passed.
  • All 70 executed CI checks passed, including Miri, ASan, TSan, platform tests, builds, and documentation. The documentation job passed after retrying a lost self-hosted runner.
  • Workspace clippy was blocked locally by the CUDA nvCOMP download; full CI linting passed.

Performance validation:

  • Three serial alternating local comparisons against the allocator-stack baseline place all 20 measured interleave, batched i8 filter, and multiplication cases within 2% of baseline.
  • The final Graviton NEON run matches develop: u64 multiplication 15.1 vs 15.3 µs; i64 and 16,384-row multiplication 17.1 vs approximately 17.3 µs. The earlier NEON slowdowns disappeared with the ZST ban.
  • The complete final CodSpeed report passed with no reported regressions.

Signed-off-by: Nicholas Gates <nick@nickgates.com>
@codspeed-hq

codspeed-hq Bot commented Sep 8, 2026

Copy link
Copy Markdown

Merging this PR will improve performance by 11.82%

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

⚡ 3 improved benchmarks
✅ 2196 untouched benchmarks
⏩ 206 skipped benchmarks1

Performance Changes

Mode Benchmark BASE HEAD Efficiency
Simulation allocate_drop_arrow[0] 456.9 ns 402.7 ns +13.45%
WallTime mul_i32_nonnull_avx512 7.9 µs 7 µs +11.64%
Simulation allocate_drop_bytes[0] 575.7 ns 521.6 ns +10.39%

Tip

Curious why performance improved? Comment @codspeedbot explain why performance improved on this PR, or directly use the CodSpeed MCP with your agent.


Comparing ngates/buffer-review-minimal (2a3f517) with develop (85b70cf)2

Open in CodSpeed

Footnotes

  1. 206 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

  2. No successful run was found on develop (5c8c397) during the generation of this report, so 85b70cf was used instead as the comparison base. There might be some changes unrelated to this pull request in this report.

Signed-off-by: Nicholas Gates <nick@nickgates.com>
@gatesn gatesn added the changelog/fix A bug fix label Sep 8, 2026
Signed-off-by: Nicholas Gates <nick@nickgates.com>
Signed-off-by: Nicholas Gates <nick@nickgates.com>
@gatesn
gatesn marked this pull request as ready for review September 8, 2026 19:00
Signed-off-by: Nicholas Gates <nick@nickgates.com>
@gatesn
gatesn marked this pull request as draft September 8, 2026 19:11
Signed-off-by: Nicholas Gates <nick@nickgates.com>
@gatesn gatesn changed the title fix(buffer): address allocator review comments with focused ZST fixes fix(buffer): reject ZSTs and address allocator review comments Sep 8, 2026
@gatesn
gatesn marked this pull request as ready for review September 8, 2026 19:27
@gatesn
gatesn requested a review from robert3005 September 8, 2026 19:44
@gatesn
gatesn enabled auto-merge (squash) September 8, 2026 19:44
@gatesn
gatesn merged commit 402fd1d into develop Sep 8, 2026
118 of 119 checks passed
@gatesn
gatesn deleted the ngates/buffer-review-minimal branch September 8, 2026 22:03
robert3005 pushed a commit that referenced this pull request Sep 9, 2026
Brings in the allocator-backed buffers (#9668, #9669, #9807) and the exponent
`Alignment` (#9730). Conflicts are resolved in favour of develop's buffer API
and internals: the tagged-word storage from this branch is re-applied under that
API in the next commit rather than merged line by line.

`Alignment` stays in `vortex-bytes`, now with develop's `u8` exponent
representation and without `Deref`. The two `scalars.rs` hunks keep
`into_vec`, which the next commit restores.

The library builds at this commit; `vortex-buffer`'s own tests and benches
target the API the next commit restores.

Signed-off-by: Claude <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Bk1yU7fmfhiFrPzF888VmT
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/fix A bug fix

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants