Skip to content

on-demand: every loadSubset forces an extra same-offset chunk request, tripping the Electric client's fast-retry-loop guard #1857

Description

@MAST1999
  • I've validated the bug against the latest version of DB packages (@tanstack/db@0.9.2, @tanstack/electric-db-collection@0.4.10, @electric-sql/client@1.5.28)

Describe the bug

Every loadSubset on an on-demand Electric collection issues an extra, non-subset chunk request at the same offset, because each one calls stream.forceDisconnectAndRefresh() before requestSnapshot.

Against a log that is not changing, those extra requests are byte-identical and repeat the same offset — which is precisely the pattern @electric-sql/client treats as a broken cache or proxy. Ten ordinary sequential subset loads are enough to trip its fast-retry-loop guard, which clears the client's caches, resets the stream, and tells the developer to go and check their proxy configuration.

The proxy is fine. The duplicate requests come from the adapter.

To Reproduce

Runnable repro against a real Postgres + Electric (the pair this repo's own e2e suite expects), so nothing fakes the wire: https://gist.github.com/MAST1999/630589ae5beb96ce5c9222a5aad003b7

docker compose up -d
npm install
node repro.mjs

It creates ten live queries with distinct where predicates, each fully awaited before the next is created. No concurrency, nothing superseded, nothing racing, and the table never changes.

distinct subsets requested   : 10
non-live shape requests      : 20
  snapshots (POST)           : 0
  chunk refetches (GET)      : 20
distinct offsets requested   : 4
requests per subset          : 2.00
subset requests              : 10

The requests alternate — ten are the subset requests themselves, and ten carry no subset__ parameters at all and repeat the same offset=0_inf:

GET ?log=changes_only&offset=now&subset__params=...&subset__where="n" >= $1&table=items
GET ?handle=...&log=changes_only&offset=0_inf&table=items
GET ?handle=...&log=changes_only&offset=0_inf&subset__params=...&subset__where=...
GET ?handle=...&log=changes_only&offset=0_inf&table=items

and the client's guard fires:

[Electric] Detected fast retry loop (5 requests in 500ms at the same offset).
Clearing client-side caches and resetting stream to recover. If this persists,
check that your proxy includes all query parameters (especially 'handle' and
'offset') in its cache key, and that required Electric headers are forwarded to
the client.

Expected behavior

Loading ten distinct subsets should cost ten subset requests. A refresh that is
only needed when the stream is actually behind should not be issued
unconditionally per subset, and ordinary usage should not trip a guard whose
message tells the developer to debug infrastructure that is working correctly.

Scope — what this repro does and does not show

It shows the request amplification (2.00 non-live requests per subset) and the spurious guard trip, both on the latest published versions.

It does not show a permanently dead stream — the client recovers in this repro. I've seen a stuck paused stream downstream of this in an application, but I could not reduce that to a minimal case, so I'm deliberately not claiming it here.

Additional context

@tanstack/db@0.9.0 removed the subset-algebra helpers (isLoadSubsetRequestSubsumedBy and friends) in favour of DeduplicatedLoadSubset sharing only exact demand identities, so a narrower predicate is no longer recognised as already covered. That makes distinct-but-overlapping predicates more common in practice, and each one now carries this extra refresh.

Related but distinct: #1855 and #1856.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions