fix(graph): stop reusing a connection across independent bulk COPY writes - #68
Open
pradeepmouli wants to merge 1 commit into
Open
Conversation
…ites copy_edges_with_bad_record_retry took a borrowed Connection and reused it across every retry attempt and the UNWIND fallback. None of these bulk loads are wrapped in an explicit transaction, so sharing the connection bought no atomicity -- it only created exposure to whatever internal state a caught COPY failure leaves behind. Observed in production: a Symbol-table COPY's bad-PK-drop-and-retry cycle left the connection wedged such that the very next COPY on it (a different table, CALLS) failed immediately with Kuzu's internal "Invalid transaction type to rollback." and fell back to the slower per-row UNWIND path. Fix: copy_edges_with_bad_record_retry now takes &GraphStore instead of &Connection and asks for a fresh connection on every retry-loop iteration and before the UNWIND fallback -- no "is this a retry" bookkeeping needed, since GraphStore::connection() already mints a fresh, cheap Connection on every call. The inline Symbol-node COPY-with-retry block in import_scip_index (a near-duplicate of the same pattern for nodes instead of edges) gets the same treatment. Threading &GraphStore down to the one caller of this helper that didn't already have it (resolve_with_map) also let resolve_inherits drop its now- entirely-unused &Connection parameter. Self-healing today via the UNWIND fallback (byte-for-byte identical output, per store_bench::test_parquet_quality), so no data-loss exposure -- but real and reproducible. Also silences one pre-existing, unrelated clippy::chunks_exact_to_as_chunks lint in embed/mod.rs (newer clippy than this branch's baseline; the workspace-wide pre-commit hook blocks on it otherwise) -- no behavior change, matches clippy's own suggested suppression.
pradeepmouli
requested review from
WinterQuant,
johnintuit,
murari316 and
sandeep-mewara
as code owners
August 25, 2026 21:09
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #67 (Bug 1 of 2 described there — the connection-reuse wedging).
What
copy_edges_with_bad_record_retrytook a borrowedConnectionand reused it across every retry attempt and theUNWINDfallback. None of these bulk loads are wrapped in an explicit transaction, so sharing the connection bought no atomicity — it only created exposure to whatever internal state a caughtCOPYfailure leaves behind.Observed in production: a
Symbol-tableCOPY's bad-PK-drop-and-retry cycle left the connection wedged such that the very nextCOPYon it (a different table,CALLS) failed immediately with Kùzu's internalInvalid transaction type to rollback.and fell back to the slower per-rowUNWINDpath. Self-healing (identical output either way, perstore_bench::test_parquet_quality), so no data-loss exposure — but real and reproducible.Fix
copy_edges_with_bad_record_retrynow takes&GraphStoreinstead of&Connectionand asks for a fresh connection on every retry-loop iteration and before theUNWINDfallback — no "is this a retry" bookkeeping needed, sinceGraphStore::connection()already mints a fresh, cheapConnectionon every call. The inline Symbol-nodeCOPY-with-retry block inimport_scip_index(a near-duplicate of the same pattern for nodes instead of edges) gets the same treatment.Threading
&GraphStoredown to the one caller of this helper that didn't already have it (resolve_with_map) also letresolve_inheritsdrop its now-entirely-unused&Connectionparameter.Also silences one pre-existing, unrelated
clippy::chunks_exact_to_as_chunkslint inembed/mod.rsthat the workspace-wide pre-commit hook caught (newer clippy than this branch's last-tested baseline) — no behavior change, matches clippy's own suggested suppression.Testing
cargo test -p infigraph-core --lib(310 tests, all pass)cargo test -p infigraph-core --test resolve_calls(15 tests, all pass — exercisesresolve_inherits/resolve_with_map)cargo test -p infigraph-core --test graph_queries(_conn-suffixed tests, all pass)cargo fmt --all -- --checkandcargo clippy --all-targets -- -D warningsboth cleanwrite_lock_perf,groups_watch_perf,index_perf) all passBug 2 from #67 (the
raw_querytransaction no-op silently breakingwrite_concerns/reflectionatomicity) is a separate, larger change (introducingGraphStore::transaction()) and will follow as its own PR.