feat: add OpenRouter quantization filter to exclude low-bit providers#11366
Draft
roomote[bot] wants to merge 2 commits intomainfrom
Draft
feat: add OpenRouter quantization filter to exclude low-bit providers#11366roomote[bot] wants to merge 2 commits intomainfrom
roomote[bot] wants to merge 2 commits intomainfrom
Conversation
Add a checkbox in OpenRouter settings to exclude low-bit quantization providers (FP4/FP6/Int4) that cause CJK encoding issues. When enabled, the provider.quantizations field is set to only allow FP8/FP16/BF16/Int8. Closes #11325
2 tasks
Contributor
Author
Reviewed all changed files across both commits. No issues found -- the implementation is clean, well-scoped, and non-breaking. The latest commit correctly adds missing translation keys to all 17 non-English locales. Mention @roomote in a comment to request specific changes to this pull request or fix all unresolved issues. |
7 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Related GitHub Issue
Closes: #11325
Description
This PR attempts to address Issue #11325. Feedback and guidance are welcome.
When using OpenRouter with CJK languages (Korean, Chinese, Japanese), low-bit quantization providers (FP4/FP6/Int4) can produce broken Unicode escape sequences instead of readable text. This adds an opt-in checkbox to filter out those providers.
Implementation details:
Schema (
packages/types/src/provider-settings.ts): AddedopenRouterExcludeLowQuantization: z.boolean().optional()to theopenRouterSchema.Handler (
src/api/providers/openrouter.ts): Extracted provider options logic into abuildProviderOptions()method that combines both specific provider routing and quantization filtering. When the setting is enabled,provider.quantizationsis set to["fp16", "bf16", "fp8", "int8"], restricting to higher-precision providers. This is applied to bothcreateMessage()andcompletePrompt().UI (
webview-ui/src/components/settings/providers/OpenRouter.tsx): Added a checkbox labeled "Exclude low-bit quantization (FP4/FP6/Int4)" with a description explaining the CJK encoding fix.i18n (
webview-ui/src/i18n/locales/en/settings.json): Added English translations for the new UI labels.Test Procedure
src/api/providers/__tests__/openrouter.spec.ts:completePrompt()as wellcd src && npx vitest run api/providers/__tests__/openrouter.spec.tsPre-Submission Checklist
Documentation Updates
Additional Notes
The setting defaults to off, preserving existing behavior. When enabled, it uses the OpenRouter documented
provider.quantizationsAPI field to restrict routing to higher-precision providers only.Important
Adds a feature to filter out low-bit quantization providers in OpenRouter settings, with schema, handler, UI, and i18n updates.
openRouterExcludeLowQuantizationtoopenRouterSchemainprovider-settings.tsto filter out low-bit quantization providers.OpenRouterHandlerinopenrouter.tsto include quantization filtering inbuildProviderOptions().OpenRouter.tsxto enable/disable the quantization filter.openrouter.spec.tsto verify quantization filtering behavior when enabled/disabled and its combination with specific provider routing.settings.json.This description was created by
for 0bb4567. You can customize this summary. It will automatically update as commits are pushed.