Skip to content

Capture and surface the local agent's reasoning as a structured, collapsible section #1330

Description

@malibio

Overview

The local ai-chat agent (Gemma 4 family) emits internal reasoning while it works — both as marked channel blocks (<|channel>thought … <channel|>) and, on weaker models, as unmarked bare-thought narration spliced into the answer text. Today the agent discards this reasoning: response_processing.rs strips marked channel blocks and tool-call prose so they never reach the user, because inline they read as garbled mid-answer rambling (e.g. …What would you like to do today?thought\nThe user responded with "Hi there"…).

That stripping is the right short-term behavior, but it throws away genuinely useful signal. The model's reasoning-toward-an-answer (why it chose create_schema over create_node, what it inferred from the request) is valuable to show — it builds trust, makes tool choices legible, and aids debugging. This issue captures the work to surface that reasoning properly instead of discarding it.

Problem Statement

Two distinct kinds of "thought" are currently lumped together and both stripped:

  1. Reasoning toward the answer (valuable): a plan/rationale the user would benefit from seeing in a labeled, collapsible section — the way reasoning-model UIs present it.
  2. Post-answer self-narration / loop chatter (noise): the model talking to itself about loop control after it already answered, an artifact of extra ReAct iterations.

We want to capture and display (1) as a first-class, structured part of the message — never inline in the answer content — while continuing to suppress (2).

The marked <|channel>…<channel|> delimiters are precisely the model signaling "this span is reasoning"; rather than deleting that span, we should route it to a dedicated reasoning field. Unmarked bare-thought narration cannot be reliably separated by regex and is better addressed by loop/model fixes (see #1329) than by display logic.

Proposed Solution

  • Parse the model's reasoning channel into a separate structured field on the assistant message (e.g. reasoning), distinct from content. Do this at the streaming/parse layer where the channel markers are still present, before they are stripped from content.
  • Plumb the reasoning field through the daemon wire format and the ai-chat node's persisted message shape.
  • Render reasoning in the chat UI as a distinct, collapsible, clearly-labeled section, visually separated from the answer. Collapsed by default.
  • Keep stripping reasoning/thought leaks from content so the answer bubble is always clean, regardless of whether reasoning capture is enabled.
  • Decide and document behavior for unmarked bare-thought narration: do NOT attempt to surface it as reasoning (unreliable to detect); rely on loop/model improvements from A/B test Gemma 4 12B (Q4_K_M) vs E4B for the local ai-chat agent #1329.

Acceptance Criteria

  • Model reasoning captured into a dedicated reasoning field on assistant messages, separate from content
  • Answer content remains free of reasoning/thought/channel leaks (existing stripping preserved)
  • reasoning plumbed through the daemon wire types and the persisted ai-chat message shape
  • Chat UI renders reasoning as a collapsible section, collapsed by default, clearly distinct from the answer
  • Behavior for unmarked bare-thought narration documented (not surfaced; deferred to model/loop fixes)
  • Tests cover: marked reasoning routed to reasoning and removed from content; clean answers with no reasoning still render normally
  • Tests pass (bun run test:all); code passes bun run quality:fix

Technical Specifications

Reference Files

  • packages/nlp-engine/src/chat/mod.rs — streaming parser + strip_gemma_special_tokens; channel markers <|channel> / <channel|> are visible here before stripping
  • packages/agent/src/local_agent/response_processing.rs — current strip_channel_blocks / strip_tool_call_prose (keep for content; reasoning capture happens upstream)
  • packages/agent/src/agent_types.rsChatMessage shape (where a reasoning field would live alongside content / tool_calls)
  • packages/daemon — wire types carrying ai-chat messages
  • packages/desktop-app/src — chat message components/types (DisplayMessage / components/chat/types.ts); render the collapsible reasoning section here

Notes

  • The reasoning channel is most reliably captured at the C++/OAI-compat parse layer, which already distinguishes channels natively; prefer that over re-parsing stripped text.
  • Frontend has multiple message representations (protocol / view-model / persisted) that intentionally do not converge — add reasoning to the appropriate layer(s) rather than forcing convergence.

Non-Goals

Related Issues

Metadata

Metadata

Assignees

Labels

Type

No type

Fields

No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions