Skip to content

chore(preview): sync the VFS engine from Effect PR #9960 - #330

Merged
lloydrichards merged 4 commits into
codex/effect-vfs-previewfrom
codex/sync-effect-vfs-preview
Nov 16, 2026
Merged

lloydrichards merged 4 commits into
codex/effect-vfs-previewfrom
codex/sync-effect-vfs-preview

Conversation

@lloydrichards

@lloydrichards lloydrichards commented Nov 16, 2026 •

Copy link
Copy Markdown
Owner

Goal/Scope

Run the existing core, memory, Node and LiveVolume consumers against the filesystem implementation now on Effect PR #9960. This PR is stacked on #329 and contains the upstream sync and removal of the migration experiments.

Description

Advance the preview from 16125827 to a70c8b9b and retire all 32 local implementation patches. Generate the preview implementation and its runtime and type suites directly from the pinned snapshot. Remove the downstream test patch and the patch application step.

Import the two new upstream storage tests. The generator now resolves imports of private filesystem modules to their corresponding preview modules so those tests exercise the same engine as core.

Remove the standalone durable-mkdir prototype and the migration diary. The core LiveVolume suite now checks reference keys, revisions and inode allocation after reopening either possible image from an unknown commit. The preview README covers workspace setup, verification and upstream refresh.

Comment

The upstream file inventory and checksums are in packages/effect-preview/upstream/manifest.json. The adapter evaluation and durability design are tracked in #328.

@vercel

vercel Bot commented Nov 16, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
effect-vfs-docs Ready Ready Preview Nov 16, 2026 12:28pm UTC

@changeset-bot

changeset-bot Bot commented Nov 16, 2026 •

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: a6845c5

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes — Reviewed the upstream engine sync, patch retirement, generator changes, and downstream compatibility.

  • Upstream sync: Advances the pinned engine to a70c8b9b and replaces 32 implementation patches with upstream code.
  • Storage safety: Rejects synchronous storage installation and borrowed-buffer reads that would persist an atime change; core already uses readAlloc.
  • Regression coverage: Retains downstream runtime/type assertions and imports upstream storage tests with correct private-module resolution.
  • Validation: bun run --cwd packages/effect-preview test passed 292 tests; bun run --cwd packages/effect-preview type-check passed compilation, 132 type assertions, and offline regeneration checks. bun run --cwd packages/core build and bun run --cwd packages/core test passed, including all 978 core tests. Memory and Node suites were not run.

Pullfrog  | View workflow run | Using gpt-6.1-sol | 𝕏

@lloydrichards
lloydrichards merged commit ea3fe51 into codex/effect-vfs-preview Nov 16, 2026
9 of 10 checks passed
@lloydrichards
lloydrichards deleted the codex/sync-effect-vfs-preview branch November 16, 2026 12:25
lloydrichards added a commit that referenced this pull request Nov 16, 2026
…view (#329)

* feat(preview): add reproducible pinned Effect VFS port

* test(core): expose single-package preview replacement blockers

* feat(core): run existing adapters on the pinned preview engine

* docs(preview): record actual runtime integration and failing acceptance

* fix(core): preserve existing behavior through preview adapters

* docs(preview): distinguish completed adapters from upstream gaps

* fix(virtual-file-system): preserve caller interruptibility during operations

Restore the caller's interruption setting while awaiting admission and at
pre-publication, so masked finalizers can complete filesystem cleanup.
Ordinary interrupted writes retain rollback before publication.

Keep the pinned snapshot unchanged. Apply a checksummed upstream-path patch
before port adaptation, including two regression tests, and verify patch
corruption in offline regeneration checks.

Both new regressions fail on the original runtime and pass with the fix.
Preview: 202 passed. Core: 694 passed, 324 failed (unchanged).
Memory: 652 passed, 70 failed (28 failures fixed; no regressions).
Node 26.11.1: 42 passed, 2 failed; mounted filesystem: 18 passed.
Preview build, type assertions, offline regeneration, lint, formatting and
knowledge validation pass. Remaining acceptance failures stay visible.

* fix(virtual-file-system): resolve terminal dot components in recursive mkdir

Resolve unguarded recursive dot paths through the existing transaction.
Retain named-slot validation for entry conditions, rooted symlink traversal,
and rollback on permission or entry-limit failures.

Add a second checksummed upstream contribution patch and ten regression
cases. Seven detect the original defect; three preserve condition behavior.
Offline regeneration now qualifies corruption checks for every patch.

Preview: 212 passed. Core: 695 passed, 323 failed (one existing failure fixed).
Memory: 652 passed, 70 failed. Node 26.11.1: 42 passed, 2 failed.
Mounted filesystem: 18 passed. No previously passing tests regress.
Build/type assertions, regeneration, lint, formatting and OKF validation pass.

* feat(virtual-file-system): support current-time attribute updates

Accept "now" in atime and mtime inputs and resolve it through the existing
transaction clock. Preserve literal/mixed timestamp ownership checks and
allow a write-authorized non-owner to request both current times. Other
fields retain prestate authorization; revisions and publication stay atomic.

Add the third checksummed upstream contribution patch with eleven runtime
cases and two public type tests. Core forwards markers instead of converting
them into literal instants. The pinned snapshot remains unchanged.

New runtime/type regressions fail without the feature and pass with it.
Preview: 223 passed. Core: 699 passed, 319 failed (four failures fixed).
Memory: 652 passed, 70 failed. Node: 42 passed, 2 failed; mounted: 18 passed.
No previously passing tests regress. Preview/core build and type checks,
regeneration, formatting, changed-file lint, and knowledge validation pass.
Core Effect diagnostics report zero errors. Remaining gaps stay visible.

* feat(virtual-file-system): support atomic resize in setAttributes

Add optional Attributes.size and stage zero-filled growth or shrinking with
metadata and timestamps in the existing transaction. Preserve prestate
authorization, revision guards, aliases, cursors, explicit times and quotas.
File handles use their open-time write grant; paths use current permission.

Keep the fourth upstream contribution reproducible through its checked patch.
Add sixteen runtime cases and two type tests, all detecting the missing feature.
Core forwards combined requests instead of rejecting or splitting them.

Preview: 239 passed. Core: 709 passed, 309 failed (ten existing failures fixed).
Memory: 652 passed, 70 failed. Node: 42 passed, 2 failed; mounted: 18 passed.
No previously passing tests regress. Preview/core build and type checks,
regeneration, changed-file lint, formatting and OKF validation pass.
Core Effect diagnostics report zero errors. Remaining gaps stay visible.

* refactor(core): replace the retained engine with the Effect preview

Delete the legacy engine, community volume/overlay implementations, watch
plumbing, adapter registry, native fallback and baseline comparison runner.
Core constructors and native binding now have one runtime implementation.
Keep public capabilities and immutable codec records outside the old engine.
Clear stale build output and audit the sole-preview import graph.

Port retained execution/ownership checks and the memory test fixture to actual
core constructors. Retire 40 passing checks of deleted private internals;
no failing test was removed and no retained test changed outcome. Fix sync
reentry and scheduler ownership exposed by running those checks on preview.

Core production source: 22,710 -> 17,088 lines; main contains 19,300.
Other production packages remain unchanged. Preserve the four reproducible
upstream patches and record remaining public behavior failures explicitly.

Validation: core 669 passed / 309 failed; memory 652/70; preview 239/0;
Node 26.11.1 provider 42/2 and mounted 18/0. Core/memory/Node builds and type
checks pass; import audit (including rejection probe), offline preview
regeneration, changed-source lint, formatting and OKF validation pass.
Core Effect diagnostics: zero errors. This remains a failing proof of concept.

* feat(virtual-file-system): support opt-in relatime

Add a checksummed upstream-path patch for strict or relatime access-time policies on make, restore and layers. Preserve strict defaults, explicit timestamps and capture records. Core selects relatime for fresh and restored volumes without emulating timestamps.

Add ten runtime regressions and one public type test. Disabling only the relatime rule fails six behavior tests while four preservation and validation tests remain passing.

Validation: preview 249 passed; core 673 passed / 305 failed; memory 653 / 69; Node provider 42 / 2; mounted 18 / 0. Exactly five existing failures recover, with no passing test regressed. Builds, types, offline regeneration, import audit, formatting, focused lint and OKF validation pass. Core Effect diagnostics report zero errors. Existing watch, persistence, overlay, atomic publication and metadata gaps remain visible.

* feat(virtual-file-system): support atomic mkdir timestamps

Add a checksummed upstream-path patch for literal mkdir preserveTimes. Snapshot and validate nested timestamps, then stage them with final-directory creation in one transaction. Preserve recursive prefixes, existing directories, parent updates and creation ctime/birthtime. Cover callers, directory-entry creation and operation files. Core forwards its existing times option and removes its directory-metadata blocker.

Eleven runtime regressions and one public type test cover timestamps, defaults, recursive budgets, byte/dot paths, conditions, quotas, permissions and cancellation. Disabling timestamp publication fails five behavior tests while six preservation/rollback tests pass.

Validation: preview 260 passed; core 674 passed / 304 failed; memory 653 / 69; Node 26.11.1 provider 42 / 2 and mounted 18 / 0. Exactly one existing failure recovers with no regressions. Builds, types, regeneration/drift/corruption checks, import audit, formatting, focused lint and OKF validation pass. Core Effect diagnostics report zero errors. Initial-open metadata and other gaps remain explicit.

* feat(virtual-file-system): support atomic open creation timestamps

Add a checksummed upstream-path patch for literal preserveTimes on creation-capable async and sync opens. Snapshot and validate nested times, then stage them in the existing open transaction before publication and handle delivery. Existing files ignore creation times; reference opens reject them. Preserve ctime/birthtime, parent changes, access grants and cursors.

Core forwards its existing times option. Correct its expectedRevision mapping for non-truncating opens after existing expected-child checks under the serialized operation. Keep revision mapping unchanged: the stale-observation test now reaches an upstream atime-only revision mismatch rather than failing during fixture creation.

Add thirteen runtime regressions and one public type test. Disabling timestamp publication fails four behavior tests while nine preservation and validation cases pass. Validation: preview 273 passed; core 674 passed / 304 failed; memory 653 / 69; Node provider 42 / 2; mounted 18 / 0. No existing test changes status. Builds, types, offline regeneration/drift/corruption checks, import audit, formatting, focused lint and knowledge validation pass. Core Effect diagnostics report zero errors. Initial size/ownership and atime-only revision neutrality remain separate gaps.

* fix(virtual-file-system): preserve revisions on read atime updates

Add a checksummed upstream-path patch that keeps automatic atime publication without advancing object revisions for atime-only touches. Explicit timestamp and other mutations retain revision advancement and guard invalidation. Document the contract and update the pinned upstream observed-read assertion.

Add seventeen strict/relatime, file/directory/symlink/handle, synchronous, guard and cancellation regressions. Removing the one-line fix fails fourteen new cases; three explicit-mutation/rollback cases still pass. Core and its test expectations are unchanged.

Validation: preview 290 passed; core 675 passed / 303 failed; memory 653 / 69; Node provider 42 / 2; mounted 18 / 0. The existing reference-open stale-observation test now passes, with no regressions. Builds, types, offline regeneration/drift/corruption checks, import audit, formatting and knowledge validation pass. Remaining gaps stay explicit.

* feat(virtual-file-system): initialize file size on creation

* feat(virtual-file-system): preserve symlink permission modes

* feat(virtual-file-system): publish bytes with exact modes and link replacement

* feat(virtual-file-system): support special modes and safe-integer owners

* feat(virtual-file-system): preserve empty symlink targets

* feat(virtual-file-system): list child identities with directory read permission

* fix(core): forward creation modes and native mkdir times

* feat(virtual-file-system): set initial ownership during open

* docs(virtual-file-system): describe patched filesystem contracts

* feat(virtual-file-system): allow parent group inheritance on creation

* fix(core): distinguish restore payload from volume content quota

* test(virtual-file-system): specify owner uid assignment authority

* style: format preview integration changes

* feat(virtual-file-system): authorize retained file attributes as the acting caller

* perf(virtual-file-system): share private file bytes on trusted capture restore

* feat(core): share private snapshot bytes with preview overlays

* fix(core): queue writers during cooperative overlay capture

* fix(core): enforce ownership for empty setattr owner updates

* fix(core): walk readable directories without touching access time

* fix(core): admit relatime reads during cooperative overlay capture

* feat(core): power volume watches from preview commits

* fix(core): defer watch subscriber wakeups past publication

* fix(core): authorize metadata changes through retained directory handles

* feat(core): project committed changes into caller-confined watches

* perf(core): avoid repeated path and grant scans during tree copy

* perf(preview): reduce path and directory lookup work in deep trees

* experiment(preview): gate staged mkdir on a storage decision

* fix(ci): build preview dependencies before consumer checks

* fix(ci): resolve the preview at runtime in benchmarks and build docs dependencies

* fix(preview): replay contribution patches with git apply

* test(preview): reopen durable mkdir decisions through core live images

* test(preview): recover both disk outcomes after an uncertain mkdir

* test(preview): persist unlinked files through final handle cleanup

* test(preview): reclaim detached directories on final handle close

* test(preview): verify SQLite recovery after writer process death

* fix(experiment): forward store durability and restore live image limits

* fix(experiment): preserve durable commit errors

* fix(experiment): drain durable commits before releasing storage

* fix(experiment): bound durable observation admission

* fix(experiment): skip storage commits for unchanged transactions

* fix(experiment): await storage when acquiring a file handle

* fix(experiment): protect preview admission from interruption

* feat(core): enable LiveVolume on the preview engine

* chore(preview): sync the VFS engine from Effect PR #9960 (#330)

* chore(preview): sync the upstream VFS engine and retain downstream regressions

* chore(preview): retire the migration experiments and document the active engine

* chore(preview): regenerate directly from the upstream snapshot

* chore(core): drop historical names from the preview import audit

* fix(persistence): preserve unknown outcomes during SQLite shutdown

* test(core): measure failure-sequence properties alone and under workspace load

* test(core): allow 15 seconds for each 100-case failure sequence

This branch was successfully deployed

1 active deployment
Preview — a6845c51 Deployed Nov 16, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant