Findings from the body-size survey behind #853 — robustness items deliberately kept out of that PR's alignment scope (the reference proxy is equally unbounded on most of these). Worth bounding on our own merits:
1. Buffered upstream responses with no cap (the request-side cap protects inbound only; these buffer whatever the upstream sends before forwarding):
jobs.rs send_upstream — every /v1/files, /v1/batches, /v1/fine_tuning call including GET /v1/files/:id/content: a multi-GB batch output file is fully buffered in RAMpassthrough.rs non-streaming responses (.bytes())jobs.rs batch-output usage-attribution downloadaudio.rs TTS / transcription result readsrerank.rs response readaisix-provider-openai non-streaming .json() success bodies
2. Upstream error bodies read with .text() before the 1 KiB truncation on the direct-HTTP handlers (messages, count_tokens, responses, rerank, audio ×2) — the bridge path already reads through read_body_capped (64 KiB); the same discipline belongs here.
3. Realtime WebSocket at library defaults, both directions: tungstenite max_message_size 64 MiB / max_frame_size 16 MiB, and max_write_buffer_size unlimited — a wedged peer can accumulate unbounded frames in the relay's write buffer. request_body_limit_bytes does not (and per #853's contract, should not) apply to WS frames.
4. Admin listener sits on axum's built-in 2 MiB body limit with the stock text/plain 413 (documented as intended in the admin OpenAPI): no envelope, no drain-before-413, no duplicate-Content-Length rejection. Revisit whether it should share the proxy's machinery.
5. Guardrail success-path .json() reads uncapped (error paths already capped at 64 KiB / 2 KiB log truncation).
None of these block #853; each is a bounded-memory improvement over both our current state and the reference proxy.
Findings from the body-size survey behind #853 — robustness items deliberately kept out of that PR's alignment scope (the reference proxy is equally unbounded on most of these). Worth bounding on our own merits:
1. Buffered upstream responses with no cap (the request-side cap protects inbound only; these buffer whatever the upstream sends before forwarding):
jobs.rssend_upstream— every /v1/files, /v1/batches, /v1/fine_tuning call includingGET /v1/files/:id/content: a multi-GB batch output file is fully buffered in RAMpassthrough.rsnon-streaming responses (.bytes())jobs.rsbatch-output usage-attribution downloadaudio.rsTTS / transcription result readsrerank.rsresponse readaisix-provider-openainon-streaming.json()success bodies2. Upstream error bodies read with
.text()before the 1 KiB truncation on the direct-HTTP handlers (messages, count_tokens, responses, rerank, audio ×2) — the bridge path already reads throughread_body_capped(64 KiB); the same discipline belongs here.3. Realtime WebSocket at library defaults, both directions: tungstenite
max_message_size64 MiB /max_frame_size16 MiB, andmax_write_buffer_sizeunlimited — a wedged peer can accumulate unbounded frames in the relay's write buffer.request_body_limit_bytesdoes not (and per #853's contract, should not) apply to WS frames.4. Admin listener sits on axum's built-in 2 MiB body limit with the stock
text/plain413 (documented as intended in the admin OpenAPI): no envelope, no drain-before-413, no duplicate-Content-Length rejection. Revisit whether it should share the proxy's machinery.5. Guardrail success-path
.json()reads uncapped (error paths already capped at 64 KiB / 2 KiB log truncation).None of these block #853; each is a bounded-memory improvement over both our current state and the reference proxy.