Skip to content

feat(buzz-agent): first-class ollama provider alias + local-models docs - #3152

Open
bitcoinaudio wants to merge 1 commit into
block:mainfrom
bitcoinaudio:feat/ollama-provider
Open

feat(buzz-agent): first-class ollama provider alias + local-models docs#3152
bitcoinaudio wants to merge 1 commit into
block:mainfrom
bitcoinaudio:feat/ollama-provider

Conversation

@bitcoinaudio

Copy link
Copy Markdown

Summary

Closes #3145.

Local models already worked through the generic OpenAI-compatible provider, but the path was undocumented and required a dummy API key plus a magic base URL. The word ollama appeared nowhere in the repo, leading operators to set model env vars on the relay (which never calls an LLM) and conclude local models were unsupported.

  • BUZZ_AGENT_PROVIDER=ollama maps onto the OpenAI-compatible path with local-friendly defaults: no API key required (a placeholder is sent, since Ollama ignores it), OPENAI_COMPAT_BASE_URL defaults to http://localhost:11434/v1
  • docs/local-models.md: where LLM config actually lives (the agent process, not the relay), Ollama quickstart, generic openai-compat recipe (vLLM / llama.cpp / LM Studio), endpoint verification, context-window tuning for small local models, desktop steps
  • crates/buzz-agent/README.md: ollama run example + env table updates

Minimal Ollama setup after this PR:

BUZZ_AGENT_PROVIDER=ollama OPENAI_COMPAT_MODEL=qwen2.5:7b-instruct ./buzz-agent

Test plan

  • 4 new unit tests: provider alias needs no key, key resolution (placeholder default / whitespace-as-unset / explicit wins / non-ollama still required), base URL defaults + override
  • cargo test -p buzz-agent — 291 lib + 65 integration tests pass
  • cargo clippy -p buzz-agent clean, cargo fmt --check clean (pre-commit/pre-push hooks green)
  • Live smoke test: real buzz-agent binary over ACP stdio (initializesession/newsession/prompt) against a local Ollama (qwen2.5:7b-instruct) with no API key set — streamed a correct reply, stopReason: end_turn

Out of scope (noted in #3145)

Desktop UI preset ("Ollama (local)" with prefilled base URL, no required key) — separate PR per the agent-config rules in desktop/src/features/agents/AGENTS.md.

Local models already worked through the generic OpenAI-compatible
provider, but the path was undocumented and required a dummy API key
plus a magic base URL. The word 'ollama' appeared nowhere in the repo,
leading operators to set model env vars on the relay (which never calls
an LLM) and conclude local models were unsupported.

- BUZZ_AGENT_PROVIDER=ollama maps onto the OpenAI-compatible path with
  local-friendly defaults: no API key required (placeholder sent),
  OPENAI_COMPAT_BASE_URL defaults to http://localhost:11434/v1
- docs/local-models.md: where LLM config actually lives (agent process,
  not the relay), Ollama quickstart, generic openai-compat recipe,
  endpoint verification, context-window tuning, desktop steps
- README: ollama run example + env table updates

Closes block#3145

Signed-off-by: bitcoinaudio <bitcoinaudio@gmail.com>

@Chessing234 Chessing234 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice thin alias — mapping ollama → openai-compat with a baked localhost base url matches how people already run local models today. docs/local-models.md is the part that was missing. for #3874 (ollama cloud) a sibling alias with https://ollama.com/v1 would reuse almost all of this path; worth a follow-up rather than stuffing cloud into this pr.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Local models (Ollama) with buzz-agent: undocumented, undiscoverable, needs first-class provider alias

2 participants