fix: GPT-5.6 SSE streams truncated through /v1/responses (chunk framing, raw identity passthrough, gzip window)

[Jiliac]

Problem

chatgpt/gpt-5.6-{sol,terra,luna} (ChatGPT-subscription Codex backend) return HTTP 200 through Plano's /v1/responses, but the client only receives response.created + response.in_progress — no text, no response.completed. Fixes #980.

Live debugging with a from-source gateway traced this to three stacked defects, each independently able to truncate or corrupt streams:

1. SSE chunk-boundary loss in SseChunkProcessor (hermesllm)

SseStreamIter splits the buffer with str::lines() and parses each line independently. When a network chunk boundary lands inside a data: /event: line prefix (e.g. da + ta: {…}), the partial line fails with a non-JSON error, which SseStreamIter::next silently skips — it never reaches the incomplete-JSON buffering. Both halves are unparseable, so the event is permanently lost with no log line. A boundary inside a multi-byte UTF-8 char could similarly kill the whole chunk via str::from_utf8.

Fix: byte-level SSE framing in process_chunk — only bytes up to the last \n are parsed; the trailing partial line is held back and prepended to the next chunk (with a completeness check for a final unterminated line like data: [DONE], since nothing flushes the buffer at end-of-stream, and a 1 MiB cap so a broken upstream can't grow the buffer unboundedly).

2. Lossy parse-and-reconstruct on the identity Responses→Responses path

The "passthrough" path strictly deserialized every event into ResponsesAPIStreamEvent and re-serialized from the struct. Any unmodeled event type or field was dropped or mangled: via-Plano streams lost reasoning.mode/reasoning.context, text.verbosity, tool_usage, …, and a future unknown event type (e.g. response.reasoning_text.delta) would be dropped entirely. Additionally, llm_gateway's per-event loop aborted the whole remaining chunk (return Err(Action::Continue)) when one event failed to parse.

Fix: on the identity Responses→Responses path, forward the original wire bytes verbatim (reconstructing the coupled event: <type>\ndata: <json>\n\n block from the raw payload — output is byte-identical to upstream). Parsing is still attempted for token counting; unknown events forward raw with provider_stream_response = None, and incomplete-JSON errors still propagate for cross-chunk recombination (shared is_incomplete_json_error helper, used by both call sites). stream_context.rs now logs and falls through on unparsed events instead of aborting the chunk. Cross-API arms and Anthropic/ChatCompletions identity behavior are untouched.

3. gzip decompressor window_bits: 9 truncates streams (envoy config)

This turned out to be the dominant user-visible cause. Envoy's decompressor filter advertises accept-encoding upstream, so the backend returns gzip. The decompressor was configured with window_bits: 9 (512-byte inflate window) while the paired compressor uses window_bits: 10 and real upstreams use 15. Gzip headers don't carry the window size, so inflate doesn't fail upfront — it silently stops emitting data once a back-reference exceeds the window. Debug trace of a failing stream: response data decompressed from 1372 bytes to 2638 bytes, then 307 → 2, then 0 for every remaining frame. This truncated any sufficiently large/redundant SSE stream traversing two listeners (gpt-5.4 included, when the client path involved compression).

Fix: decompressor window_bits: 15 on all four gzip decompressor blocks (three were at 9; the egress_traffic_llm one had no window_bits and silently used Envoy's default of 12). Note the blast radius: this config applies to all providers' gzip-encoded responses, not just chatgpt — which is intended, since the truncation class affects any of them. Raising the inflate window is strictly safe (a larger window decodes anything a smaller one can; max_inflate_ratio zip-bomb protection is retained).

Also: added gpt-5.6-{sol,terra,luna} to the chatgpt models in provider_models.yaml (consumed via include_str! in ProviderId), and a pre-existing one-line clippy fix (for_kv_map) needed to keep the workspace -D warnings gate green.

Verification

Unit tests (all inline, ids obfuscated):

• test_gpt56_responses_identity_passthrough_full_stream — replays a real captured GPT-5.6 stream (11 events incl. a reasoning output item) through the processor at every 64-byte split offset, asserting all event types emerge and output bytes == input bytes. Failed before the fix; passes now.
• Mid-prefix split, mid-JSON recombination, unknown-event tolerance, bounded-buffer, and serde relaxation tests (OutputItem::Reasoning without summary / with content/encrypted_content/status; omitted logprobs).
• Full gates: cargo test --lib (388 tests across brightstaff/common/hermesllm/llm_gateway/prompt_gateway), cargo fmt --check, cargo clippy --locked --all-targets --all-features -- -D warnings, wasm32-wasip1 release builds.

Live through a from-source gateway (planoai build + up):

• All three chatgpt/gpt-5.6-* tiers and chatgpt/gpt-5.4 now stream the full ladder (created … output_text.delta … completed) with correct answer text.
• Previously-dropped fields (reasoning.mode, text.verbosity, tool_usage) are preserved byte-for-byte.

Notes

• Wire-format change: identity Responses→Responses streams are now verbatim upstream bytes instead of Plano's re-serialized shape — strictly more spec-faithful; scoped to that identity pair only.
• Orthogonal to Stoff81/chatgpt5.5 #958 (ChatGPT/5.5 path): this change deliberately doesn't touch the ResponsesAPIStreamBuffer cross-API path Stoff81/chatgpt5.5 #958 works in.

🤖 Generated with Claude Code