Dsv4 parsing encoding fixes#4
Open
tarruda wants to merge 2 commits into
Open
Conversation
DeepSeek V4 uses the same DSML invoke and parameter markup as DeepSeek V3.2, but the outer call block is named tool_calls instead of function_calls. The generic autoparser can infer parts of this format, but it is not compatible with DeepSeek V4 reasoning semantics: it treats the reasoning prefix as an optional closed block, so a partial generation that has opened <think> but has not emitted </think> can still fall through into tool parsing. That lets DSML-looking text inside the thinking stream be interpreted as an executable tool call. DeepSeek only intends DSML tool calls to be actionable after the thinking block has closed. DwarfStar handles this with a manual parser, and the same behavior is needed here to avoid running tool calls produced inside unfinished reasoning and to preserve parallel DSML invokes. Reuse the existing DeepSeek DSML parser for V4 by parameterizing the outer block name. The parser now accepts tool_calls in addition to function_calls, handles the V4 no-thinking </think> prefill, keeps schema enforcement for real tool calls after reasoning closes, supports repeated invokes when parallel tool calls are enabled, and treats an unclosed <think> block as reasoning-only. Add DeepSeek V4 parser coverage for plain content, a single DSML tool call, parallel DSML tool calls, and DSML markup inside an unclosed reasoning block. Validation: git diff --cached --check; build/bin/test-chat. Assisted-by: Codex
DeepSeek V4 encoding documents several tool conversation invariants that the current template and specialized parser were close to, but did not fully match. First, previous assistant reasoning is part of tool-call history. The V4 template only preserved reasoning for assistant turns after the last user message, so an assistant turn with tool calls could be rendered without its reasoning once tool results and a follow-up user message were present. Keep reasoning whenever tools are available, while preserving the existing continuation behavior for non-tool conversations. Second, tool results are encoded inside user turns and must be ordered according to the preceding assistant tool_calls array. OpenAI-compatible clients can deliver role:tool messages in completion order rather than call order, so normalize contiguous tool result messages before rendering DeepSeek DSML templates. Third, DSML parameters are name-tagged, but the specialized PEG parser required all required parameters first in schema order and only allowed optional parameters afterward. That rejected valid model output with optional parameters before required parameters, even though the parameter names make the order unambiguous. Build the argument grammar from the set of remaining required parameters so parameters can appear in any order while still requiring each required parameter to be present. Add DeepSeek V4 coverage for optional-before-required parsing, missing required parameter rejection, continuation prompts, reasoning preservation across tool calls, and tool result ordering. Validation: cmake --build build --target test-chat -j 8; git diff --check; build/bin/test-chat --template DeepSeek-V4; build/bin/test-chat. Assisted-by: Codex
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two commits to fix parsing/encoding. More context in this reddit thread