-
Notifications
You must be signed in to change notification settings - Fork 29
feat: real end-to-end SSE / StreamingResponse over Zig HTTP server (closes #163)#164
feat: real end-to-end SSE / StreamingResponse over Zig HTTP server (closes #163) #164justrach wants to merge 1 commit into
Conversation
Closes #163. Sub-task of #146. ## What was broken EventSourceResponse and StreamingResponse were Python facades only. The Zig HTTP server had no awareness of streaming: dispatch read response.body directly, which is b"" for streaming responses, so SSE responses arrived with empty bodies (status 200, content-type text/event-stream, Content-Length 0). Verified with the wire-level probe in #163 comments. ## What this PR does Adds a chunked-transfer write path through a small ABI extension: * New 5-tuple ABI for streaming: (status, content_type, b"", iterator, headers_dict). Existing 3-tuple stays unchanged for non-streaming. * python/turboapi/responses.py — StreamingResponse.body_iter() returns a sync iterator. Sync sources wrapped directly. Async sources go through _AsyncToSyncChunkIterator which drives the worker thread's event loop one chunk at a time, so SSE / token streams flush in real time without blocking. * python/turboapi/request_handler.py — five dispatch sites (pos, async pos, fast noargs, fast, async fast) detect StreamingResponse and return the 5-tuple. * python/turboapi/async_pool.py — _normalize_response_tuple (the load-bearing path for async dispatch via awaitPythonCoroutineResponse) also detects StreamingResponse and returns the 5-tuple. * zig/src/server.zig — new sendStreamingResponse: writes the response head with Transfer-Encoding: chunked + custom headers from the dict (filtering ones that conflict with framing), loops PyIter_Next on the iterator and writes each as <hex-len>\\r\\n<bytes>\\r\\n, terminates with 0\\r\\n\\r\\n. Skips zero-length yields. Clears Python error state on StopIteration. Streaming responses are explicitly not cacheable. ## Wire-level proof Re-running the #163 probe (5-event SSE generator, async handler, 50ms between events) — before this PR all 5 events disappeared into an empty 200 response. After this PR: 5 chunks delivered at ~50ms intervals via Transfer-Encoding: chunked, TTFB 4ms, TTLB 259ms. Custom headers (Cache-Control: no-cache, X-Accel-Buffering: no) reach the wire. ## CI tests tests/test_sse_e2e.py — 6 tests booting a real subprocess server and hitting it with httpx + raw socket: chunked encoding, custom headers, real-time streaming, terminator framing, raw bytes generators. All 6 pass. 44-test regression suite still green. ## What's NOT done * WebSocket: separate next PR (#114). * create_enhanced_handler (slow / Depends path) StreamingResponse hookup — rare combination; follow-up. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 9458dc296f
i️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
P1 Badge Do not terminate errored generators as successful streams
When a streaming iterator raises after headers or some chunks have already been sent, PyIter_Next returns NULL with a Python exception set; this branch clears that exception and then writes the normal 0\r\n\r\n terminator. In an SSE/token stream this makes upstream failures look like a clean end-of-stream to clients, so partial/corrupt output can be accepted as complete instead of surfacing an incomplete chunked response or at least logging the generator error.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
P1 Badge Reject CRLF in streamed response headers
Streaming responses now copy headers_dict directly into the HTTP header block, but neither the header name nor value is checked for \r/\n before formatting. If an app reflects any user-controlled value into StreamingResponse(..., headers=...) or EventSourceResponse(..., headers=...), this permits HTTP response splitting/header injection on the wire; skip or reject such headers before appending them.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
✅ Devin Review: No Issues Found
Devin Review analyzed this PR and found no potential bugs to report.
View in Devin Review to see 6 additional findings.
Uh oh!
There was an error while loading. Please reload this page.
Closes #163. Sub-task of #146 (the user building a ChatGPT-style backend). Pairs with the SSE wire-format unit tests merged in #162.
What was broken
EventSourceResponseandStreamingResponsewere Python facades only. The Zig HTTP server had no awareness of streaming: dispatch readresponse.bodydirectly, which isb\"\"for streaming responses, so SSE responses arrived with empty bodies (status 200,Content-Type: text/event-stream,Content-Length: 0). Verified with the wire-level probe documented in #163.Fix
Adds a chunked-transfer write path through a small ABI extension.
Wire-level proof
Re-running the #163 probe (5-event SSE generator, async handler, 50ms between events).
Before this PR — all 5 events disappeared into an empty 200 response (`Content-Length: 0`).
After this PR — raw socket capture:
```
HTTP/1.1 200 OK
Server: TurboAPI
Date: 2026年5月10日 14:06:04 GMT
Content-Type: text/event-stream
Transfer-Encoding: chunked
Connection: keep-alive
cache-control: no-cache
x-accel-buffering: no
10\r\ndata: {"i": 0}\n\n\r\n
10\r\ndata: {"i": 1}\n\n\r\n
10\r\ndata: {"i": 2}\n\n\r\n
10\r\ndata: {"i": 3}\n\n\r\n
10\r\ndata: {"i": 4}\n\n\r\n
0\r\n\r\n
```
httpx-stream view: 5 chunks delivered at ~50ms intervals, TTFB 4ms, TTLB 259ms (matches the `asyncio.sleep` cadence — chunks flush in real time, not pre-buffered).
CI tests
`tests/test_sse_e2e.py` — 6 tests booting a real subprocess server on port 18765 and hitting it with httpx + raw socket:
All 6 pass locally. 44-test regression suite (test_async_handlers, test_request_parsing, test_post_body_parsing, test_returns_model_cache*, test_annotated_depends) all green.
What's NOT done in this PR
Checklist
🤖 Generated with Claude Code