Skip to content

feat(messages): cross-provider content blocks beyond text — image / tool_use / tool_result / document #722

Description

@jarvis9443

Summary

/v1/messages cross-provider translation is text-only on the request side: image, tool_use, tool_result, document, and thinking content blocks are silently dropped when the target is a non-Anthropic upstream (crates/aisix-provider-anthropic/src/wire.rs:976-984; the module docstring at wire.rs:900-901 already promises this follow-up). Consequence: multi-turn tool loops (e.g. Claude Code against a non-Anthropic model) lose their tool history, and vision/document inputs vanish. Matrix row "Anthropic Messages 跨厂商 = 仅文本块" in api7/AISIX-Cloud#873 section ⑤.

Scope (map exactly to LiteLLM LiteLLMAnthropicMessagesAdapter.translate_anthropic_messages_to_openai, litellm/llms/anthropic/experimental_pass_through/adapters/transformation.py:353)

Request → OpenAI chat shape:

  • image (base64 → data: URL; url → passthrough) → image_url content part
  • document/PDF → file content part (image_url-style per LiteLLM)
  • assistant tool_usetool_calls[] entry (name 64-char truncation, arguments JSON-encoded)
  • tool_resultrole:"tool" message keyed by tool_use_id; string content, single-block, and multi-block (text+image combined) forms
  • thinking/redacted_thinking history blocks → dropped for OpenAI-wire upstreams (same effective behavior as LiteLLM's OpenAI provider transform; document it)

Response → Anthropic shape: text + tool_use already work (non-stream + stream); verify nothing regresses and add coverage.

Tests

  • Unit tests on the wire translation match arms per block type.
  • DP e2e: anthropic-shape multi-turn tool loop against a mock OpenAI upstream (tool_use + tool_result history preserved); vision request round-trip.

Part of the section-⑤ gap closure for api7/AISIX-Cloud#873.

Metadata

Metadata

Assignees

No one assigned

    Labels

    P2Long-tail integrations — backlogenhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions