Symptom
When an upstream OpenAI Chat SSE response emits partial content and then closes without a non-null finish_reason or a [DONE] marker, Switchyard treats EOF as a clean completion. The target encoder can then synthesize a successful terminal event, so downstream clients cannot distinguish truncation from success.
Reproduction
The minimal regression uses switchyard-translation directly and requires no provider credentials:
#[test]
fn decode_stream_rejects_eof_before_a_terminal_event() -> Result<(), BoxError> {
let sse = b"data: {"choices":[{"delta":{"content":"partial"},"finish_reason":null}]}
".to_vec();
let bytes = stream::once(async move { Ok::<Vec, LlmClientError>(sse) });
let results =
block_on(decode_stream(bytes, WireFormat::OpenAiChat)?.collect::<Vec<_>>());
assert!(results.first().is_some_and(Result::is_ok));
let Some(Err(LlmClientError::ResponseTranslation(message))) = results.last() else {
panic!("expected incomplete OpenAI stream to fail");
};
assert_eq!(message, "openai_chat stream ended before a terminal event");
Ok(())
}
On unmodified main at a17efa9:
test helpers::tests::decode_stream_rejects_eof_before_a_terminal_event ... FAILED
expected incomplete OpenAI stream to fail
The same shape was observed through switchyard-server 0.2.0: partial assistant text was delivered with HTTP 200 and no failure signal.
Expected vs. actual
- Expected: EOF before a protocol terminal event produces a stream error. Already-delivered partial chunks may remain visible, but Switchyard must not synthesize a successful terminal event or [DONE].
- Actual: decode_stream ends normally after the partial event. encode_stream finalization can synthesize a stop/completed event and the server can emit [DONE].
Environment
- Switchyard version (or commit SHA): reproduced on main at a17efa9; originally observed on switchyard-server 0.2.0
- Python version: not applicable; native Rust path
- OS / arch: reproduced test on Linux x86_64 under WSL2
- Install path: source build with Rust 1.96.1
- Inbound format: OpenAI Chat Completions
- Backend: controlled OpenAI-compatible loopback upstream
Additional context
The correct ownership appears to be switchyard-translation::decode_stream, before target-format encoding. The decoder knows the source wire format and can require one of:
- OpenAI Chat: a non-null finish_reason or [DONE]
- OpenAI Responses: response.completed, response.incomplete, or [DONE]
- Anthropic Messages: message_stop (an optional [DONE] must not substitute for it)
In-band provider error events remain terminal failures and should not receive a second EOF error.
Related: #283 addresses premature Anthropic EOF specifically. A protocol-aware source-terminal invariant would cover that case and the OpenAI false-success path without adding heuristics in the server or downstream clients.
Symptom
When an upstream OpenAI Chat SSE response emits partial content and then closes without a non-null finish_reason or a [DONE] marker, Switchyard treats EOF as a clean completion. The target encoder can then synthesize a successful terminal event, so downstream clients cannot distinguish truncation from success.
Reproduction
The minimal regression uses switchyard-translation directly and requires no provider credentials:
".to_vec();
let bytes = stream::once(async move { Ok::<Vec, LlmClientError>(sse) });
let results =
block_on(decode_stream(bytes, WireFormat::OpenAiChat)?.collect::<Vec<_>>());
On unmodified main at a17efa9:
The same shape was observed through switchyard-server 0.2.0: partial assistant text was delivered with HTTP 200 and no failure signal.
Expected vs. actual
Environment
Additional context
The correct ownership appears to be switchyard-translation::decode_stream, before target-format encoding. The decoder knows the source wire format and can require one of:
In-band provider error events remain terminal failures and should not receive a second EOF error.
Related: #283 addresses premature Anthropic EOF specifically. A protocol-aware source-terminal invariant would cover that case and the OpenAI false-success path without adding heuristics in the server or downstream clients.