Non-streaming /v1/chat/completions leaks thinking into content when truncated (finish_reason: length) before </think>

[stefanwichmann]

Summary

In non-streaming chat completions, if generation is cut off by max_tokens
(i.e. finish_reason: "length") before the model emits its closing </think>,
the entire partial thinking block is returned as message.content and
message.reasoning_content is absent/empty.

When the same request is allowed to finish (finish_reason: "stop"), the split
is correct: the thinking lands in reasoning_content and content holds only
the final answer. Streaming is also correct — even a truncated stream emits the
partial thinking as reasoning_content deltas and never as content.

So the leak is specific to the blocking (non-streaming) response path when the
</think> boundary never arrives. Any client that sets a modest max_tokens
will intermittently receive raw chain-of-thought presented as the assistant's
answer.

Environment

• ds4 built from main (CUDA target), ~2026-07-06
• NVIDIA GB10 (DGX Spark), CUDA 13.0
• Model: deepseek-v4-flash, Q2 imatrix quant
• Default thinking mode, no tools in the request

Reproduction

Same prompt in all three cases; only max_tokens / stream differ.

A) Non-streaming, truncated — BUG

curl -s http://127.0.0.1:8000/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"model":"deepseek-v4-flash","messages":[{"role":"user","content":"What is 17 * 24?"}],"max_tokens":32,"temperature":0}'

Observed:

finish_reason:     length
reasoning_content: None
content:           'We need to compute 17 * 24. This is a simple multiplication. 17 * 24 = 17 * (20 + 4) ='

The thinking is in content; reasoning_content is missing.

B) Non-streaming, allowed to finish — correct

curl -s http://127.0.0.1:8000/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"model":"deepseek-v4-flash","messages":[{"role":"user","content":"What is 17 * 24?"}],"max_tokens":512,"temperature":0}'

Observed:

finish_reason:     stop
reasoning_content: "We need to compute 17 * 24. ... So answer is 408."
content:           'The product of 17 and 24 is 408.'

C) Streaming, truncated — correct (for contrast)

Same body as (A) plus "stream": true. Observed over the SSE stream:

finish_reason=length, reasoning_content deltas=31, content deltas=0

The partial thinking is correctly labelled reasoning_content, even though the
</think> boundary is never reached.

Expected

Non-streaming responses should classify tokens the same way streaming does. When
a response is truncated mid-thought:

• the partial thinking should be returned in reasoning_content, and
• content should be empty (there is no answer yet),

so that finish_reason: "length" responses are consistent with both the
completed non-streaming case (B) and the streaming case (C).

Probable cause

The behaviour is consistent with the blocking JSON path buffering the full
generation and then splitting on the </think> terminator to populate
reasoning_content vs content. If generation stops before </think> is
emitted, there is no terminator to split on, so the whole buffer falls through
to content.

This is the non-streaming analogue of #13 ("/v1/chat/completions streams
thinking into content instead of reasoning_content when no tools are
requested"). #13 fixed the streaming path by routing non-tool chats through
the structured streamer that already handles thinking; the blocking path
appears to still lack the equivalent "unterminated thinking → reasoning_content"
handling.

Relationship to existing issues

• /v1/chat/completions streams thinking into content instead of reasoning_content when no tools are requested #13 (closed) — streaming variant of this leak; fixed. Streaming now
  behaves correctly (case C confirms). This report is the non-streaming
  counterpart that the /v1/chat/completions streams thinking into content instead of reasoning_content when no tools are requested #13 fix did not cover.
• Unclosed </think> before tool call stalls agent session (finish=stop, no tool call delivered) #318 (open) — the tool-call variant of a missing/unclosed </think>
  boundary (server drops the DSML call, finish=stop, nothing useful returned).
  Same underlying theme: partial/unterminated thinking is not handled
  gracefully. A shared fix for "treat an unterminated thinking block as
  reasoning" would likely address both.
• ds4-server chat thinking benchmarks include hidden checkpoint rebuild after length stop #76 (closed) — also involves length-stopped responses, but concerns
  benchmark wall-time (a post-generation checkpoint rebuild), not the
  reasoning/content classification. Mentioned only to disambiguate.

Impact

• deepseek-v4-flash emits substantial thinking even for trivial prompts, so
  the truncation window is easy to hit with ordinary max_tokens values.
• Downstream proxies/clients that rely on the reasoning_content field to
  separate thinking from the answer (rather than parsing <think> tags out of
  content) will surface raw chain-of-thought as the answer whenever a
  non-streaming response is length-capped.
• The behaviour is inconsistent across three otherwise-equivalent code paths
  (streaming vs blocking, truncated vs completed), which makes it a subtle
  correctness trap.

Suggested fix

In the blocking response builder, when the generation ends without a </think>
terminator (e.g. finish_reason == "length" while still inside the thinking
block), treat the accumulated text as reasoning_content and leave content
empty — mirroring the streaming classifier and the #13 fix.