fix(proxy): translate Anthropic tools/tool_choice to OpenAI shape in cross-provider dispatch (streaming + non-streaming) - #254
Merged
Conversation
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughAdds translators to convert Anthropic ChangesTool translation for cross-provider dispatch
Sequence DiagramsequenceDiagram
participant Client
participant Gateway as cross_provider_dispatch
participant Translators as translate_anthropic_*_to_openai
participant OpenAIUpstream
Client->>Gateway: POST /v1/messages (Anthropic tools/tool_choice)
Gateway->>Gateway: Detect upstream is non-Anthropic
Gateway->>Translators: translate_anthropic_tools_to_openai
Translators-->>Gateway: OpenAI-shaped tools
Gateway->>Translators: translate_anthropic_tool_choice_to_openai
Translators-->>Gateway: OpenAI tool_choice
Gateway->>OpenAIUpstream: Dispatch with converted shapes
OpenAIUpstream-->>Gateway: {tool_calls:[...]}
Gateway-->>Client: Anthropic-shaped tool_use response
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Possibly related issues
Possibly related PRs
🚥 Pre-merge checks | ✅ 4✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@crates/aisix-proxy/src/messages.rs`:
- Around line 410-418: The code removes "tools" and "tool_choice" from
chat.extra before attempting translation, which causes silent loss when
translation returns None; update the logic in the block handling
translate_anthropic_tools_to_openai and
translate_anthropic_tool_choice_to_openai so you don't remove the original entry
until translation succeeds — e.g., peek at chat.extra (using get or
remove_then_reinsert pattern) to obtain the value, call
translate_anthropic_tools_to_openai/translate_anthropic_tool_choice_to_openai,
and only replace (remove + insert) when Some(translated) is returned; if
translation returns None, leave the original entry untouched in chat.extra (or
explicitly reinsert the original) to avoid dropping the field.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 80517cbf-9f41-4fe1-971b-5eb8d11fa8dd
📒 Files selected for processing (4)
crates/aisix-provider-anthropic/src/lib.rscrates/aisix-provider-anthropic/src/wire.rscrates/aisix-proxy/src/messages.rstests/e2e/src/cases/anthropic-tools-cross-provider-e2e.test.ts
There was a problem hiding this comment.
Pull request overview
Fixes cross-provider tool calling for the Anthropic Messages endpoint (POST /v1/messages) when the selected upstream is OpenAI-compatible by translating tools and tool_choice from Anthropic shape into the OpenAI tools schema before dispatch.
Changes:
- Add Anthropic → OpenAI translators for
toolsandtool_choiceinaisix-provider-anthropic. - Invoke these translators in the
/v1/messagescross-provider dispatch path so non-Anthropic bridges receive OpenAI-shaped fields. - Add unit coverage for both translators and an E2E test validating the full Anthropic client → OpenAI upstream round-trip (including response tool-call translation back to Anthropic).
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| tests/e2e/src/cases/anthropic-tools-cross-provider-e2e.test.ts | New E2E test ensuring Anthropic-shaped tools/tool_choice are translated to OpenAI shape on the upstream request and tool calls translate back on the response. |
| crates/aisix-proxy/src/messages.rs | Applies the new translations in cross_provider_dispatch() for non-Anthropic upstreams. |
| crates/aisix-provider-anthropic/src/wire.rs | Implements Anthropic → OpenAI translation helpers and adds unit tests for them. |
| crates/aisix-provider-anthropic/src/lib.rs | Re-exports the new translation helpers for use by the proxy crate. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
jarvis9443
force-pushed
the
fix/issue-236-anthropic-tools-to-openai
branch
from
May 12, 2026 01:42
849e01b to
d6fb334
Compare
jarvis9443
force-pushed
the
fix/issue-236-anthropic-tools-to-openai
branch
from
May 12, 2026 01:50
d6fb334 to
7c6cb3a
Compare
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@crates/aisix-provider-anthropic/src/wire.rs`:
- Around line 804-821: The current code uses unwrap_or("") for id/name and
accepts any parsed arguments, which can emit invalid "tool_use" entries; change
the logic that processes each tc so it only pushes a "tool_use" JSON when
tc.get("id") yields a non-empty string, tc.get("function").get("name") yields a
non-empty string, and the parsed arguments from
tc.get("function").get("arguments") successfully deserialize to a
serde_json::Value that is an object (e.g., Value::is_object()); otherwise skip
that tc entry entirely. Locate the variables id, name, input and the
content.push call in this block and replace the unconditional unwrap_or behavior
with guarded checks (if let / match) that continue/skip malformed entries before
constructing the tool_use object.
- Around line 314-319: The code currently copies optional fields directly into
the function map (using t.get("description")/t.get("input_schema") and
function.insert), which can forward non-string/non-object values; update the
logic to validate types first: only insert "description" when the retrieved desc
is a string and only insert "parameters" when the retrieved schema is a JSON
object/map/struct (i.e., the expected input_schema shape); if the type check
fails, silently skip the field to preserve lenient behavior and avoid producing
invalid OpenAI tools payloads.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 18f3ee19-9835-40ab-bcd4-e7b85f3e1e67
📒 Files selected for processing (4)
crates/aisix-provider-anthropic/src/lib.rscrates/aisix-provider-anthropic/src/wire.rscrates/aisix-proxy/src/messages.rstests/e2e/src/cases/anthropic-tools-cross-provider-e2e.test.ts
🚧 Files skipped from review as they are similar to previous changes (2)
- crates/aisix-provider-anthropic/src/lib.rs
- tests/e2e/src/cases/anthropic-tools-cross-provider-e2e.test.ts
jarvis9443
force-pushed
the
fix/issue-236-anthropic-tools-to-openai
branch
from
May 12, 2026 01:58
7c6cb3a to
965c1a9
Compare
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@crates/aisix-provider-openai/src/wire.rs`:
- Around line 189-210: The streaming conversion path currently only copies
role/content and drops message.tool_calls, so streamed tool-use events are lost;
update the stream-handling code that builds ChatMessage (the same logic that
calls role_from_str and finish_reason and constructs ChatMessage with
content_blocks/name/tool_call_id/extra) to mirror this non-stream branch: create
or reuse a serde_json::Map extra, check c.message.tool_calls (or the stream
chunk equivalent) and, if present and non-empty, insert "tool_calls" ->
Value::Array(tool_calls) into extra, then assign that extra to ChatMessage.extra
so streamed responses propagate tool_calls the same way as the non-stream path.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 8f93a045-7e1a-470b-9ab0-da76edaab897
📒 Files selected for processing (5)
crates/aisix-provider-anthropic/src/lib.rscrates/aisix-provider-anthropic/src/wire.rscrates/aisix-provider-openai/src/wire.rscrates/aisix-proxy/src/messages.rstests/e2e/src/cases/anthropic-tools-cross-provider-e2e.test.ts
🚧 Files skipped from review as they are similar to previous changes (4)
- crates/aisix-proxy/src/messages.rs
- crates/aisix-provider-anthropic/src/lib.rs
- tests/e2e/src/cases/anthropic-tools-cross-provider-e2e.test.ts
- crates/aisix-provider-anthropic/src/wire.rs
jarvis9443
force-pushed
the
fix/issue-236-anthropic-tools-to-openai
branch
from
May 12, 2026 02:06
965c1a9 to
d820d24
Compare
…cross-provider dispatch (#236) When a client sends an Anthropic Messages API request with tools and tool_choice to a non-Anthropic upstream, the gateway now translates these fields from Anthropic shape to OpenAI shape before dispatching. Previously, tools ({name, description, input_schema}) and tool_choice ({type: "any"}) were passed through verbatim in ChatFormat.extra, which OpenAI-compatible upstreams silently ignored or rejected. Added: - translate_anthropic_tools_to_openai(): converts Anthropic tool defs to OpenAI {type:"function", function:{name, description, parameters}} - translate_anthropic_tool_choice_to_openai(): converts Anthropic tool_choice ({type:"any"} → "required", {type:"tool",name} → {type:"function",function:{name}}, etc.) - Called in cross_provider_dispatch() after parse_inbound_request() - Unit tests for both translators - E2E test for Anthropic client → OpenAI upstream tool round-trip Closes #236
jarvis9443
force-pushed
the
fix/issue-236-anthropic-tools-to-openai
branch
from
May 12, 2026 02:13
d820d24 to
84c4320
Compare
…vider dispatch Extends #236 fix to handle streaming mode: - Add tool_calls field to ChatDelta (gateway core) and OpenAiStreamDelta - Propagate streaming tool_calls through stream_chunk_into_chat_chunk - Update AnthropicSseEncoder with per-tool-call state machine that emits content_block_start (tool_use), content_block_delta (input_json_delta), and content_block_stop events - Update RenderedDelta to forward tool_calls in OpenAI streaming path - Add unit tests for streaming encoder (single tool, multi-tool, mixed text+tool, force_finish with open tool blocks) - Add E2E test for streaming tool_calls round-trip
jarvis9443
changed the title
Contributor
Author
|
CI status: lint, unit tests, build, coverage all pass. E2E failures are pre-existing flaky guardrail tests ( |
This was referenced May 14, 2026
This was referenced May 18, 2026
Closed
This was referenced Jul 3, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
When a client sends an Anthropic Messages API request with
toolsandtool_choiceto a non-Anthropic upstream, the gateway now translates these fields from Anthropic shape to OpenAI shape before dispatching — in both streaming and non-streaming modes.Previously, tools (
{name, description, input_schema}) and tool_choice ({type: "any"}) were passed through verbatim inChatFormat.extra, which OpenAI-compatible upstreams silently ignored or rejected.Request-side (both modes)
translate_anthropic_tools_to_openai(): converts Anthropic tool defs to OpenAI{type:"function", function:{name, description, parameters}}translate_anthropic_tool_choice_to_openai(): converts{type:"any"}→"required",{type:"tool", name}→{type:"function", function:{name}}, etc.cross_provider_dispatch()afterparse_inbound_request()Response-side — non-streaming
OpenAiResponseMessagenow deserializestool_callsand propagates toChatMessage.extrachat_response_into_anthropic_json()translates OpenAItool_calls→ Anthropictool_usecontent blocksResponse-side — streaming
OpenAiStreamDeltanow deserializestool_callsand propagates toChatDeltaAnthropicSseEncoderupdated with per-tool-call state machine that emits:content_block_startwithtype: tool_use(id, name)content_block_deltawithtype: input_json_delta(argument fragments)content_block_stopfor each open block on finishRenderedDeltaupdated to forward tool_calls in OpenAI streaming pathTests
Closes #236