Problem
execute_sse_request retries with the same request_body in two cases that
should be terminal:
- A TTFT (time-to-first-token) timeout re-sends an already-sent request. The
request body is sent to the server before the first-token wait begins, so
the retry re-sends an already-billed request (double-billing for the same
logical turn).
- A deterministic 4xx (for example 400) is retried. A deterministic client
error fails identically on every attempt, so the retry only burns the
request budget/credits and adds latency.
Root cause
The retry decision in execute_sse_request does not distinguish terminal from
transient outcomes. The TTFT branch continues using the same request body,
and the non-2xx branch retries every status, including 400. The
SEND_MESSAGE_STREAM_ATTEMPTS counter is amplified by these re-sends.
Repro
- For TTFT: mock a server that never completes a response, inject a small
ttft_timeout, and observe execute_sse_request re-send the same body
across attempts.
- For 4xx: mock a server that returns 400 and observe the fixture is called
more than once (the old bad_requests_then_success fixture even reports
success when a later attempt returns 200).
Impact
Billing is charged twice for the same logical turn on TTFT timeout, and
deterministic client errors waste credits/latency across up to
SEND_MESSAGE_STREAM_ATTEMPTS re-sends.
Problem
execute_sse_requestretries with the samerequest_bodyin two cases thatshould be terminal:
request body is sent to the server before the first-token wait begins, so
the retry re-sends an already-billed request (double-billing for the same
logical turn).
error fails identically on every attempt, so the retry only burns the
request budget/credits and adds latency.
Root cause
The retry decision in
execute_sse_requestdoes not distinguish terminal fromtransient outcomes. The TTFT branch
continues using the same request body,and the non-2xx branch retries every status, including 400. The
SEND_MESSAGE_STREAM_ATTEMPTScounter is amplified by these re-sends.Repro
ttft_timeout, and observeexecute_sse_requestre-send the same bodyacross attempts.
more than once (the old
bad_requests_then_successfixture even reportssuccess when a later attempt returns 200).
Impact
Billing is charged twice for the same logical turn on TTFT timeout, and
deterministic client errors waste credits/latency across up to
SEND_MESSAGE_STREAM_ATTEMPTSre-sends.