feat(messages): cross-provider content blocks beyond text — image / tool_use / tool_result / document - #726
Conversation
…on cross-provider dispatch Pre-#722 the /v1/messages inbound parse kept only text blocks when the target Model was non-Anthropic: image, document, tool_use, tool_result and thinking blocks were silently dropped, so a Claude-Code-style multi-turn tool loop lost its tool history (empty user turns upstream) and vision/document inputs vanished. The translation now follows LiteLLM's anthropic adapter map (litellm/llms/anthropic/experimental_pass_through/adapters/ transformation.py): - image (base64 -> data: URL, url -> passthrough) and document -> OpenAI image_url content parts, riding ChatMessage.content_blocks (forwarded verbatim by the openai bridge); content keeps the concatenated text for guardrail scans - assistant tool_use -> OpenAI tool_calls[] (JSON-encoded arguments, 64-char name cap); a pure tool-call turn gets content: null - user tool_result -> a role:"tool" message keyed by tool_use_id, emitted BEFORE the user turn (OpenAI ordering); string / single-text / multi-block (text+image) content forms - thinking / redacted_thinking history blocks drop -- not replayable on the OpenAI wire; LiteLLM's OpenAI provider transform discards them the same way (top-level thinking still maps to reasoning_effort) Response direction (text + tool_use, non-stream + stream) already worked and is regression-covered. Fixes #722
|
Warning Review limit reachedYou’ve reached a temporary PR review limit under our Fair Usage Limits Policy. Next review available in: 23 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (4)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
1 participant
Summary
Closes the "Anthropic Messages 跨厂商 = 仅文本块" row of AISIX-Cloud#873 §⑤:
/v1/messagesrequests routed to a non-Anthropic Model now translate the full content-block set instead of silently dropping everything but text. The user-visible failure this fixes: a Claude-Code-style multi-turn tool loop lost its tool history on cross-provider dispatch (the upstream saw empty user turns), and vision/document inputs vanished —wire.rseven documented it as "land in a follow-up PR". This is that follow-up.Fixes #722
Block map (LiteLLM
LiteLLMAnthropicMessagesAdapterparity)text{type:"text"}part / plain string (unchanged)imagebase64image_urlpart withdata:<media>;base64,…URLimageurlimage_urlpart, URL passthroughdocumentimage_urldata-URL part (LiteLLM's_translate_anthropic_image_to_openaimapping)tool_usetool_calls[]entry — JSON-encodedarguments, 64-char name cap (LiteLLMtruncate_tool_name); a pure tool-call turn getscontent: nulltool_resultrole:"tool"message keyed bytool_use_id, emitted before the user turn (OpenAI requires tool answers directly after the assistanttool_callsturn); string / single-text-collapse / multi-block (text+image parts) content formsthinking/redacted_thinkingthinkingkey still maps toreasoning_effort)Mechanics: multimodal parts ride the existing
ChatMessage.content_blocks(already forwarded verbatim by the openai bridge — the same array vision uses on/v1/chat/completions), so no bridge changes were needed;contentkeeps the concatenated text so guardrail scans still see the message. Anthropic-upstream passthrough is untouched (byte-for-byte as before). Response direction (text + tool_use blocks, non-stream + stream) already worked — regression-covered.Tests
anthropic-content-blocks-cross-provider-e2e.test.ts(real binary + etcd + mock OpenAI upstream) pinning the translated upstream wire: full tool-loop history (user → assistant(tool_calls) → tool → user) and vision data-URL round-trip./v1/messagesproxy tests (39), anthropic crate tests (90) and the cross-provider matrix / tools e2e all green.