Skip to content

[bug] Reject upstream SSE that ends before a terminal event #424

Description

@yoliverasPozo

Symptom

When an upstream OpenAI Chat SSE response emits partial content and then closes without a non-null finish_reason or a [DONE] marker, Switchyard treats EOF as a clean completion. The target encoder can then synthesize a successful terminal event, so downstream clients cannot distinguish truncation from success.

Reproduction

The minimal regression uses switchyard-translation directly and requires no provider credentials:

#[test]
fn decode_stream_rejects_eof_before_a_terminal_event() -> Result<(), BoxError> {
    let sse = b"data: {"choices":[{"delta":{"content":"partial"},"finish_reason":null}]}

".to_vec();
let bytes = stream::once(async move { Ok::<Vec, LlmClientError>(sse) });
let results =
block_on(decode_stream(bytes, WireFormat::OpenAiChat)?.collect::<Vec<_>>());

    assert!(results.first().is_some_and(Result::is_ok));
    let Some(Err(LlmClientError::ResponseTranslation(message))) = results.last() else {
        panic!("expected incomplete OpenAI stream to fail");
    };
    assert_eq!(message, "openai_chat stream ended before a terminal event");
    Ok(())
}

On unmodified main at a17efa9:

test helpers::tests::decode_stream_rejects_eof_before_a_terminal_event ... FAILED
expected incomplete OpenAI stream to fail

The same shape was observed through switchyard-server 0.2.0: partial assistant text was delivered with HTTP 200 and no failure signal.

Expected vs. actual

  • Expected: EOF before a protocol terminal event produces a stream error. Already-delivered partial chunks may remain visible, but Switchyard must not synthesize a successful terminal event or [DONE].
  • Actual: decode_stream ends normally after the partial event. encode_stream finalization can synthesize a stop/completed event and the server can emit [DONE].

Environment

  • Switchyard version (or commit SHA): reproduced on main at a17efa9; originally observed on switchyard-server 0.2.0
  • Python version: not applicable; native Rust path
  • OS / arch: reproduced test on Linux x86_64 under WSL2
  • Install path: source build with Rust 1.96.1
  • Inbound format: OpenAI Chat Completions
  • Backend: controlled OpenAI-compatible loopback upstream

Additional context

The correct ownership appears to be switchyard-translation::decode_stream, before target-format encoding. The decoder knows the source wire format and can require one of:

  • OpenAI Chat: a non-null finish_reason or [DONE]
  • OpenAI Responses: response.completed, response.incomplete, or [DONE]
  • Anthropic Messages: message_stop (an optional [DONE] must not substitute for it)

In-band provider error events remain terminal failures and should not receive a second EOF error.

Related: #283 addresses premature Anthropic EOF specifically. A protocol-aware source-terminal invariant would cover that case and the OpenAI false-success path without adding heuristics in the server or downstream clients.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions