Bound benchmark ingestion retries and validate before transactions - #9799
Bound benchmark ingestion retries and validate before transactions#9799connortsui20 wants to merge 1 commit into
Conversation
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
Merging this PR will improve performance by 24.24%
|
| Mode | Benchmark | BASE |
HEAD |
Efficiency | |
|---|---|---|---|---|---|
| ⚡ | WallTime | arrow_checked_add_u32_neon[16384] |
20.4 µs | 13.3 µs | +53.13% |
| ⚡ | Simulation | allocate_drop_arrow[0] |
456.9 ns | 402.7 ns | +13.45% |
| ⚡ | Simulation | allocate_drop_bytes[0] |
575.7 ns | 521.6 ns | +10.39% |
Tip
Curious why performance improved? Comment @codspeedbot explain why performance improved on this PR, or directly use the CodSpeed MCP with your agent.
Comparing ct/bench-writer-bounds (b1b5bfc) with develop (85b70cf)2
Footnotes
-
206 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩
-
No successful run was found on
develop(5c8c397) during the generation of this report, so 85b70cf was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩
Summary
Validates the commit SHA and every record before starting the ingest transaction, then retries only PostgreSQL deadlocks and serialization failures. Retries stop after eight attempts or a 120-second elapsed budget, which prevents further attempts without cancelling an active transaction.
Changes
Adds jittered backoff, connection/statement/lock timeouts, and attempt diagnostics while preserving whole-file upserts, frozen measurement IDs, and best-effort cache refresh. Adds CI coverage against the website's actual migrations, with 24 passing tests covering validation, retry bounds, real database conflicts, atomic rollback, replay, and refresh failure.