Skip to content

fs: add macOS F_RDADVISE + micro-prefetch to accelerate SwiftCompile reads#2348

Closed
erneestoc wants to merge 2 commits into
TraceMachina:mainfrom
erneestoc:ec/pr2243-macos-f-rdadvise
Closed

fs: add macOS F_RDADVISE + micro-prefetch to accelerate SwiftCompile reads#2348
erneestoc wants to merge 2 commits into
TraceMachina:mainfrom
erneestoc:ec/pr2243-macos-f-rdadvise

Conversation

@erneestoc

@erneestoc erneestoc commented May 18, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds macOS `F_RDADVISE` + sequential-read hints + page-cache warming to the worker's read paths. This is the read-side complement to the write-side optimizations in #2338's APFS `clonefile(2)` work — and the macOS equivalent of the Linux `POSIX_FADV_SEQUENTIAL` hint that's already used on this codepath.

For a Bazel SwiftCompile p95 input shape (~466 MB, ~1,978 files), the read side dominates wall time. `F_RDADVISE` tells the macOS kernel "I'm about to read this file sequentially" — enables aggressive readahead and removes per-syscall stalls.

What changes

File Change
`nativelink-util/src/fs.rs` New `FileSlot::advise_sequential()` and `FileSlot::advise_willneed(offset, len)` helpers. macOS impl uses `libc::fcntl(F_RDADVISE)`. Linux impl uses existing `posix_fadvise` route. All other targets are `const fn` no-ops.
`nativelink-store/src/filesystem_store.rs` `get_part` calls `advise_sequential` once on open; micro-prefetches the next 2 chunks inside the read loop.
`nativelink-util/src/store_trait.rs` Same pair of hooks in `slow_update_store_with_file`.
`nativelink-worker/src/directory_cache.rs` Fire-and-forget `warm_page_cache` after each successful hardlink cache hit (read path), via `tokio::spawn` with a bounded depth.

Implementation notes

  • macOS's `F_RDADVISE` constant (`44`) and `radvisory` struct layout are not exposed by the `libc` crate. Both are declared locally with `#[repr(C)]` — verified against `<sys/fcntl.h>` on macOS 14+.
  • cfg-gating is per-method so the three implementations of each name coexist cleanly: macOS gets the real syscall; Linux falls through to `posix_fadvise`; other targets are no-op `const fn`s.
  • The page-cache warming on the DirectoryCache read path is fire-and-forget by design — it's a hint, not a correctness operation, and we don't want to block the materialize path waiting for it.

Tests

  • `fs::advise_tests::macos_f_rdadvise_returns_success` — macOS-only, exercises the syscall path against a real `FileSlot` fd and asserts the kernel returned success.
  • `fs::advise_tests::advise_helpers_are_callable_everywhere` — cross-platform sanity that the no-op compiles and returns successfully on Linux/Windows.

All 7 existing `fs` and `fs_util` tests pass.

Verification

  • `cargo build -p nativelink-util -p nativelink-store -p nativelink-worker` clean
  • `cargo test -p nativelink-util --lib` 7/7 pass; `cargo test -p nativelink-worker --lib` 3/3 pass
  • `cargo clippy -p nativelink-util -p nativelink-store -p nativelink-worker --lib --tests -- -D warnings` clean for changed files (note: the 2 pre-existing clippy errors in `nativelink-store/src/r2_store.rs` from Replace bincode with wincode #2345 — `new_ret_no_self`, `single_match_else` — also fire on plain `origin/main`; unrelated to this PR)
  • `cargo fmt --check` clean

Relationship to other work

🤖 Generated with Claude Code


This change is Reviewable

erneestoc and others added 2 commits May 18, 2026 16:03
Adds two best-effort kernel page-cache hints to FileSlot:

  * `advise_sequential()` — Linux uses `posix_fadvise(SEQUENTIAL)`,
    macOS issues `F_RDADVISE` over the first 4 MiB to kick off
    readahead (the documented analogue), no-op elsewhere.
  * `advise_willneed(offset, len)` — Linux uses
    `posix_fadvise(WILLNEED)`, macOS uses `fcntl(F_RDADVISE)` with a
    locally-declared `radvisory` struct (not exposed by the `libc`
    crate), no-op elsewhere.

Wired into the read path:

  * `filesystem_store::get_part` calls `advise_sequential` after the
    file is opened and micro-prefetches the next two buffer-sized
    chunks inside the read loop so kernel readahead overlaps with the
    network send.
  * `slow_update_store_with_file` does the same when streaming a
    local file into a store.
  * `directory_cache::get_or_create` fires a tokio task after each
    successful cache-hit hardlink to walk the freshly-materialized
    tree and `advise_willneed` every file, warming the page cache
    before the action that triggered the materialization starts
    reading.

For the macOS Mac-Mini iOS workload (SwiftCompile dominates, p95
input ~466 MB) this is a direct attack on read-side wall time and
complements the clonefile work landed in PR TraceMachina#2338. The Linux paths
use the existing `posix_fadvise` infrastructure for symmetry.

Tests: cross-platform smoke test that the helpers compile to
no-op-or-real on every target, plus a macOS-only test that drives
the raw `F_RDADVISE` fcntl directly and asserts it returns 0.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replace raw tokio::spawn / tokio::task::spawn_blocking calls in
directory_cache.rs with background_spawn! and spawn_blocking! from
nativelink-util::task. The raw calls were tripping the workspace
clippy::disallowed_methods lint and bypassing the tracing spans /
OpenTelemetry context propagation the wrappers provide.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@erneestoc erneestoc closed this May 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant