[Rust] Reject server over-acks - #648
Conversation
Track the highest physical offset handed to each gRPC connection and reject malformed cumulative acknowledgements before they can advance durability state. Preserve pending single records and batches on protocol violations. Fixes databricks#638 Signed-off-by: sama Pyb <peiyibopybsama@gmail.com>
|
Revalidated this change against current Current-main evidence:
One current-main OAuth timing test, I also tested composition with adjacent PR #792 ( An independent concurrency review found 0 Critical and 0 Important issues and gave GO. The conflict-resolved tree is staged locally only; I have not pushed a new commit or made a new DCO/signature attestation, so this PR remains Draft. |
What changes are proposed in this pull request?
Fixes #638.
This change makes the Rust gRPC receiver reject malformed cumulative durability acknowledgements before they mutate client state. It:
A landing-zone observed count is not a safe upper bound because
observe()happens before the bounded outbound channel handoff. A connection-local sent watermark closes that concurrency window and is reset together with the physical offset generator on recovery.How is this tested?
cargo test -p databricks-zerobus-ingest-sdk(162 unit tests plus 24 passing doc tests; 9 doc tests ignored by existing annotations)cargo test -p tests --test rust_tests(93 integration tests)reserve()does not advance the sent watermark1 -> 0 -> 2sequencecargo clippy -p databricks-zerobus-ingest-sdk --all-targets -- -D warningsmake checkgit diff --check