fix(core): guard concurrent local document sidecar acceptance

[phernandez]

Problem

The local document importer in #1529 checks sidecar provenance and source generation before writing, but those checks are separate from filesystem replacement. Two concurrent-write edge cases remain:

1. A user edits an existing sidecar (or creates a file at a previously absent sidecar path) after the last check and before FileService.write_file() replaces it. The unconditional replacement can discard those canonical bytes.
2. Import A checks source generation A, then pauses. The source changes and import B accepts generation B. When A resumes, it can recognize B's sidecar as an untouched generated projection and replace it with generation A. Indexing and the watcher do not rerun extraction, so another import is needed to repair it.

Atomic file replacement does not make the preceding checksum check atomic with replacement.

Scope and decision

Follow-up to #1529, deferred by the maintainer so the Office/CSV ingestion work can land. Keep the current ordinary refresh behavior in that PR. This issue tracks stronger acceptance semantics; it does not claim the existing preflight checks provide compare-and-swap.

Review evidence:

• Sidecar replacement race
• Stale source generation race

Acceptance criteria

• Define the supported concurrency boundary explicitly, including separate importer processes and external editors; a process-local lock alone does not protect arbitrary editor writes.
• Preserve user-created or edited canonical sidecar bytes when they arrive between validation and acceptance, or use a documented conflict-preserving behavior when a platform cannot provide conditional replacement.
• An older extraction cannot roll back a sidecar already accepted from a newer source generation; it must conflict or no-op.
• Add deterministic interleaving tests for edit-before-replace, create-before-first-write, and old-import-after-new-import acceptance, including Windows filesystem behavior.
• Preserve run-ledger provenance and normal unchanged-import reuse/source-change refresh. Avoid wider database locking for derived index state.

[claude]

Claude finished @phernandez's task in 36s —— View job

---

Triage

Type: Bug — a real concurrency defect in existing code. The preflight checks added in #1529 (sidecar provenance/checksum, source generation) run separately from the actual file replacement in FileService.write_file(), so the check-then-act sequence isn't atomic. Two concrete races are named: (1) a user-edited or newly-created sidecar between the check and the unconditional replace can be discarded, and (2) a stale import (older source generation) that resumes after a newer import already accepted can roll the sidecar back, with no automatic repair since indexing/watch don't rerun extraction.

Component: Local document importer / core sync (FileService, import pipeline). Not cloud-specific — labeled component: none per the repo's label taxonomy, since this affects local filesystem writes regardless of cloud sync.

Priority: Medium. This is a real correctness gap (canonical bytes can be silently discarded, or a stale extraction can undo a newer one), but it's a narrow timing window requiring concurrent imports or an edit landing in a specific gap, and it was consciously deferred by the maintainer in #1529 so other ingestion work could land first. No user-reported data loss yet — this is proactive hardening.

Complexity: Complex. The acceptance criteria call for a compare-and-swap-style atomic replace (or a documented conflict-preserving fallback where the platform can't do that, notably Windows), explicit definition of the supported concurrency boundary (external editors vs. process-local locks vs. separate importer processes), a generation-aware "no rollback from an older extraction" invariant, and new deterministic interleaving tests — while explicitly avoiding wider database locking per the repo's derived-state consistency model. This is architectural work on the write path, not a quick fix.

Duplicate check: No existing open or closed issue covers this. Related work: #1529 (the PR that introduced the checks this issue hardens) and #1512 (the docx/pptx/csv ingestion feature this was deferred in favor of) — both referenced for context, neither a duplicate.

Labels applied: bug.

[phernandez]

Candidate implementation on branch feat/1530-note-lock-fence (one commit on top of #1529's af3ffea, not rebased onto 7245d5c): guarded_note_replacement holds a filelock advisory lock per note (under the system temp dir so the indexer never indexes it) and re-checks the note checksum under the lock, with None meaning the note must still be absent. Inside that critical section the writer calls the reader's require_generation so an import whose source moved on stops before it can roll a newer sidecar back. Both notes (sidecar and run note) use the primitive. Tests cover the stale-import ordering, creation-over-a-gap, lock contention, and the fence itself; the module stays at 100% coverage.