feat(buzz-agent): first-class ollama provider alias + local-models docs - #3152
Open
bitcoinaudio wants to merge 1 commit into
Open
feat(buzz-agent): first-class ollama provider alias + local-models docs#3152bitcoinaudio wants to merge 1 commit into
bitcoinaudio wants to merge 1 commit into
Conversation
Local models already worked through the generic OpenAI-compatible provider, but the path was undocumented and required a dummy API key plus a magic base URL. The word 'ollama' appeared nowhere in the repo, leading operators to set model env vars on the relay (which never calls an LLM) and conclude local models were unsupported. - BUZZ_AGENT_PROVIDER=ollama maps onto the OpenAI-compatible path with local-friendly defaults: no API key required (placeholder sent), OPENAI_COMPAT_BASE_URL defaults to http://localhost:11434/v1 - docs/local-models.md: where LLM config actually lives (agent process, not the relay), Ollama quickstart, generic openai-compat recipe, endpoint verification, context-window tuning, desktop steps - README: ollama run example + env table updates Closes block#3145 Signed-off-by: bitcoinaudio <bitcoinaudio@gmail.com>
Chessing234
reviewed
Jul 31, 2026
Chessing234
left a comment
Contributor
There was a problem hiding this comment.
nice thin alias — mapping ollama → openai-compat with a baked localhost base url matches how people already run local models today. docs/local-models.md is the part that was missing. for #3874 (ollama cloud) a sibling alias with https://ollama.com/v1 would reuse almost all of this path; worth a follow-up rather than stuffing cloud into this pr.
3 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes #3145.
Local models already worked through the generic OpenAI-compatible provider, but the path was undocumented and required a dummy API key plus a magic base URL. The word
ollamaappeared nowhere in the repo, leading operators to set model env vars on the relay (which never calls an LLM) and conclude local models were unsupported.BUZZ_AGENT_PROVIDER=ollamamaps onto the OpenAI-compatible path with local-friendly defaults: no API key required (a placeholder is sent, since Ollama ignores it),OPENAI_COMPAT_BASE_URLdefaults tohttp://localhost:11434/v1docs/local-models.md: where LLM config actually lives (the agent process, not the relay), Ollama quickstart, generic openai-compat recipe (vLLM / llama.cpp / LM Studio), endpoint verification, context-window tuning for small local models, desktop stepscrates/buzz-agent/README.md: ollama run example + env table updatesMinimal Ollama setup after this PR:
Test plan
cargo test -p buzz-agent— 291 lib + 65 integration tests passcargo clippy -p buzz-agentclean,cargo fmt --checkclean (pre-commit/pre-push hooks green)buzz-agentbinary over ACP stdio (initialize→session/new→session/prompt) against a local Ollama (qwen2.5:7b-instruct) with no API key set — streamed a correct reply,stopReason: end_turnOut of scope (noted in #3145)
Desktop UI preset ("Ollama (local)" with prefilled base URL, no required key) — separate PR per the agent-config rules in
desktop/src/features/agents/AGENTS.md.