Skip to content

fix(graph): stop reusing a connection across independent bulk COPY writes - #68

Open
pradeepmouli wants to merge 1 commit into
intuit:mainfrom
pradeepmouli:fix/graphstore-connection-transaction-ownership
Open

fix(graph): stop reusing a connection across independent bulk COPY writes#68
pradeepmouli wants to merge 1 commit into
intuit:mainfrom
pradeepmouli:fix/graphstore-connection-transaction-ownership

Conversation

@pradeepmouli

Copy link
Copy Markdown
Contributor

Fixes #67 (Bug 1 of 2 described there — the connection-reuse wedging).

What

copy_edges_with_bad_record_retry took a borrowed Connection and reused it across every retry attempt and the UNWIND fallback. None of these bulk loads are wrapped in an explicit transaction, so sharing the connection bought no atomicity — it only created exposure to whatever internal state a caught COPY failure leaves behind.

Observed in production: a Symbol-table COPY's bad-PK-drop-and-retry cycle left the connection wedged such that the very next COPY on it (a different table, CALLS) failed immediately with Kùzu's internal Invalid transaction type to rollback. and fell back to the slower per-row UNWIND path. Self-healing (identical output either way, per store_bench::test_parquet_quality), so no data-loss exposure — but real and reproducible.

Fix

copy_edges_with_bad_record_retry now takes &GraphStore instead of &Connection and asks for a fresh connection on every retry-loop iteration and before the UNWIND fallback — no "is this a retry" bookkeeping needed, since GraphStore::connection() already mints a fresh, cheap Connection on every call. The inline Symbol-node COPY-with-retry block in import_scip_index (a near-duplicate of the same pattern for nodes instead of edges) gets the same treatment.

Threading &GraphStore down to the one caller of this helper that didn't already have it (resolve_with_map) also let resolve_inherits drop its now-entirely-unused &Connection parameter.

Also silences one pre-existing, unrelated clippy::chunks_exact_to_as_chunks lint in embed/mod.rs that the workspace-wide pre-commit hook caught (newer clippy than this branch's last-tested baseline) — no behavior change, matches clippy's own suggested suppression.

Testing

  • cargo test -p infigraph-core --lib (310 tests, all pass)
  • cargo test -p infigraph-core --test resolve_calls (15 tests, all pass — exercises resolve_inherits/resolve_with_map)
  • cargo test -p infigraph-core --test graph_queries (_conn-suffixed tests, all pass)
  • cargo fmt --all -- --check and cargo clippy --all-targets -- -D warnings both clean
  • Pre-commit hook's perf suite (write_lock_perf, groups_watch_perf, index_perf) all pass

Bug 2 from #67 (the raw_query transaction no-op silently breaking write_concerns/reflection atomicity) is a separate, larger change (introducing GraphStore::transaction()) and will follow as its own PR.

…ites

copy_edges_with_bad_record_retry took a borrowed Connection and reused it
across every retry attempt and the UNWIND fallback. None of these bulk
loads are wrapped in an explicit transaction, so sharing the connection
bought no atomicity -- it only created exposure to whatever internal state
a caught COPY failure leaves behind. Observed in production: a Symbol-table
COPY's bad-PK-drop-and-retry cycle left the connection wedged such that the
very next COPY on it (a different table, CALLS) failed immediately with
Kuzu's internal "Invalid transaction type to rollback." and fell back to
the slower per-row UNWIND path.

Fix: copy_edges_with_bad_record_retry now takes &GraphStore instead of
&Connection and asks for a fresh connection on every retry-loop iteration
and before the UNWIND fallback -- no "is this a retry" bookkeeping needed,
since GraphStore::connection() already mints a fresh, cheap Connection on
every call. The inline Symbol-node COPY-with-retry block in
import_scip_index (a near-duplicate of the same pattern for nodes instead
of edges) gets the same treatment.

Threading &GraphStore down to the one caller of this helper that didn't
already have it (resolve_with_map) also let resolve_inherits drop its now-
entirely-unused &Connection parameter.

Self-healing today via the UNWIND fallback (byte-for-byte identical output,
per store_bench::test_parquet_quality), so no data-loss exposure -- but
real and reproducible.

Also silences one pre-existing, unrelated clippy::chunks_exact_to_as_chunks
lint in embed/mod.rs (newer clippy than this branch's baseline; the
workspace-wide pre-commit hook blocks on it otherwise) -- no behavior
change, matches clippy's own suggested suppression.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Connection/transaction lifecycle bugs in KuzuBackend: wedged COPY retries + silently non-atomic writes

1 participant