fix(provider): enable image support for custom OpenAI-compatible providers#21627
fix(provider): enable image support for custom OpenAI-compatible providers#21627jwcrystal wants to merge 1 commit into
Conversation
|
Thanks for updating your PR! It now meets our contributing guidelines. 👍 |
015bdbe to
4d93919
Compare
|
@adamdotdevin review this please when you are available |
|
@rekram1-node Please review this. I’ll really appreciate it because the image isn’t being loaded by OpenAI-compatible providers. |
…iders Custom OpenAI-compatible providers were unable to process image attachments because the default capabilities.input.image was set to false when modalities were not explicitly declared in the provider configuration. This change enables image support by default for openai-compatible providers.
2797d19 to
27c6586
Compare
|
Hi! I noticed this PR addresses the same issue from a different angle. My PR #26826 fixes image support at the protocol conversion layer in The root cause I found: Both PRs address the same goal but at different layers - #21627 at the provider capability detection layer, #26826 at the protocol conversion layer. They're complementary and together provide a complete fix. My PR also includes comprehensive unit tests (16 tests passing) covering media handling scenarios. |
Issue for this PR
Closes #20802
Type of change
What does this PR do?
Custom OpenAI-compatible providers (e.g., longent, Ollama with vision models) were unable to process image attachments. Users received the error: "I can't read `image.png` here because this model does not support image input" even when the provider actually supported vision capabilities.
Root cause: In
packages/opencode/src/provider/provider.ts:1110, the default value forcapabilities.input.imagewasfalsefor custom providers without explicitmodalitiesdeclaration. This causedtransform.ts:265to filter out image parts before they could reach the existing conversion logic inconvert-to-openai-compatible-chat-messages.ts.Fix: Changed the default to
truespecifically for providers using@ai-sdk/openai-compatible:This is a one-line change that:
modalities.input.image(highest priority)trueinstead offalseWhy this works: The existing conversion logic in
convert-to-openai-compatible-chat-messages.ts:45-64already correctly transformsfileparts toimage_urlformat. The issue was that custom providers were being flagged as "not supporting image input" before the conversion could happen.How did you verify your code works?
modalitiesdeclarationmodalitiesstill use their declarationChecklist