perf(codex): pace WebSocket connects adaptively instead of at a fixed 1s

[momomuchu]

Problem

Spacing fresh WebSocket connection starts (529e1d6, "limit codex websocket connection churn") protects an origin from upgrade-rejection bursts. The protection is right, but it is billed continuously rather than on demand.

codex_previous_response_id() defaults to false, so continuation is off and there is no pool owner for an ordinary request. Every generation therefore opens a fresh socket and pays the spacing. WS_CONNECT_GATE is a process-global Lazy<WebSocketConnectGate> whose wait_to_start sleeps to previous + 1s while holding the mutex, so the spacing is not per-connection — it serializes the whole process.

The result is a hard ceiling of roughly one generation per second per process, independent of how healthy the origin is and of any concurrency the caller configures.

Measured

One process, idle account, 12 trivial /v1/messages requests, all issued concurrently:

1 process   13.27s   0.90 req/s
2 processes  7.64s   1.57 req/s

Throughput tracked the number of processes, not the number of in-flight requests — the signature of a process-global serializer.

Change

Spacing becomes the price of an observed rejection instead of a standing tax:

• starts at the configured floor (Duration::ZERO by default),
• widens to 1s on an origin-forbidden upgrade and doubles up to 8s,
• halves back toward the floor after 20 consecutive successful connects.

connect_with_policy now reports each outcome to the gate (note_origin_forbidden / note_connect_success). The existing single 403 retry-after-cooldown is unchanged, and the pre-existing connect_gate_* locks still pass because a gate constructed with an explicit spacing keeps that spacing as its floor.

Both signal paths use try_lock and return on contention: losing a widen signal is harmless (the next rejection carries it) whereas blocking there would stall the connect path.

Operators who want guaranteed pacing set a floor the gate never relaxes below, via CCP_CODEX_WS_CONNECT_SPACING_MS or codex.websocketConnectSpacingMs.

Result

Controlled A/B, same account, same window, 30 concurrent requests against one process:

	fixed 1s	adaptive
wall	37.35s	7.18s
throughput	0.80 req/s	4.18 req/s
success	28/30	30/30

5.2x on a single process, with fewer failures rather than more.

A 12-request burst shows no difference between the two — at that size the per-request latency masks the gate. The regression only becomes visible once the burst exceeds the latency-bandwidth product, which is worth knowing for anyone re-measuring this.

Deployed locally across 12 processes afterwards: bursts of 10/30/60 concurrent requests completed 176/176 with no errors, and throughput kept rising with concurrency (no saturation up to 60), where previously it was pinned near 1 req/s.

Tests

New unit locks in src/providers/codex/websocket.rs:

• idle_gate_starts_connections_without_spacing_them
• gate_spaces_starts_after_the_origin_rejects_an_upgrade
• repeated_rejections_widen_the_spacing_up_to_a_cap
• sustained_success_relaxes_the_spacing_back_to_zero
• a_configured_floor_is_never_relaxed_away

The last two are the negative controls: without relaxation a single 403 would tax the process for its whole life, and without a respected floor an operator's explicit pacing would be optimised away.

just check-ci green on the commit: ✓ format-check ✓ clippy ✓ build ✓ test.

Scope

Codex WebSocket connect pacing only. No change to continuation, pooling, model routing, request translation, other providers, or the 403 retry contract.