Skip to content

feat(desktop): auto-enable huddle transcription for agents - #3180

Merged
tlongwell-block merged 11 commits into
mainfrom
jtennant/auto-transcribe-agent-huddles
Jul 31, 2026
Merged

feat(desktop): auto-enable huddle transcription for agents#3180
tlongwell-block merged 11 commits into
mainfrom
jtennant/auto-transcribe-agent-huddles

Conversation

@johnmatthewtennant

@johnmatthewtennant johnmatthewtennant commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

Context

Before this change, every huddle initialized with transcription off. Joining or adding an agent did not enable it, so the agent could not receive spoken conversation until a person clicked the transcript control. Starting a huddle from an agent DM could also omit that agent, and adding an agent who already belonged to the parent channel could attempt an unnecessary role change and show a warning.

Agent detection uses authoritative huddle membership. A participant counts as an agent when the ephemeral membership identifies it with the bot role, or when the existing agent identity model identifies the participant in an agent DM.

Summary

Buzz now enables transcription once when the first authoritative agent is present. After that initial automatic action, explicit user control is authoritative: manual ON or OFF survives membership refreshes, reconnects, and UI remounts. Removing the last agent does not change the current transcription state.

Agent-DM huddles enroll the agent automatically. Adding an agent who already belongs to the parent channel preserves the existing parent role and completes without a role-mutation warning.

Scenario Before With this change
First authoritative agent joins or is hydrated Transcription stays off Transcription turns on once
User explicitly turns transcription on or off Manual control exists without an agent policy The explicit choice suppresses later automatic changes
Last agent leaves No defined agent-presence behavior The current transcription state remains unchanged
Huddle starts from an agent DM The agent can be omitted The known agent is enrolled automatically
Added agent already belongs to the parent channel Buzz can attempt a role rewrite and warn Existing parent membership and role are preserved
Transcription is active The control is not visually distinct The control is highlighted and exposes aria-pressed=true

Changes

  • Derive agent presence from authoritative bot-role huddle membership and known agent-DM identity.
  • Apply the one-time auto-enable rule during create, join, membership hydration, reconnect, pipeline startup, and local agent addition.
  • Preserve explicit user state and use huddle-generation guards so stale asynchronous work cannot alter a replacement huddle.
  • Keep backend and React transcription state synchronized, with a visible and accessible active control.
  • Enroll known agent-DM participants and make parent-channel membership updates idempotent.
  • Cover hydration ordering, reconnects, remounts, explicit OFF, last-agent removal, DM enrollment, existing membership, and active styling.

Related issue

None found.

Testing

Manual validation in pending-seed confirmed the product contract:

  1. Started a huddle from the owned, running Fizz agent DM.
  2. Confirmed the authoritative roster contained the human and Fizz as an agent.
  3. Confirmed transcription enabled without clicking the control: Stop transcript, aria-pressed=true, with the highlighted active background.
  4. Turned transcription off and confirmed Start transcript, aria-pressed=false remained stable.
  5. Removed Fizz while transcription was off and confirmed the state stayed off.
  6. Left the huddle cleanly.

Screenshots

The same control has distinct active and inactive states.

Active transcript control

Inactive transcript control

Reviewer-reproducible examples

From a fresh checkout:

pnpm --dir desktop build:e2e
pnpm --dir desktop exec playwright test tests/e2e/huddle-transcription.spec.ts --project=smoke
pnpm --dir desktop exec playwright test tests/e2e/mentions.spec.ts --project=smoke --grep "system agent profile exposes owned agent actions|system agent avatar exposes owned agent actions|owned bot profile exposes message and huddle actions|owned agent mention profile exposes message and huddle actions"

The huddle scenario exercises initial authoritative hydration, exactly one automatic enable, explicit OFF persistence, unchanged state after last-agent removal, newer events winning over delayed hydration, agent-DM enrollment, and idempotent parent membership. It also asserts aria-pressed and distinct computed active styling.

@johnmatthewtennant
johnmatthewtennant marked this pull request as ready for review July 27, 2026 19:18
@johnmatthewtennant
johnmatthewtennant requested a review from a team as a code owner July 27, 2026 19:18
@johnmatthewtennant
johnmatthewtennant force-pushed the jtennant/auto-transcribe-agent-huddles branch from 67b4842 to 55a674c Compare July 27, 2026 19:51
@johnmatthewtennant
johnmatthewtennant marked this pull request as draft July 27, 2026 19:53
@johnmatthewtennant
johnmatthewtennant force-pushed the jtennant/auto-transcribe-agent-huddles branch 5 times, most recently from 7b152da to 2ff30f3 Compare July 28, 2026 19:31
Signed-off-by: John Tennant <jtennant@squareup.com>
Signed-off-by: John Tennant <jtennant@squareup.com>
Signed-off-by: John Tennant <jtennant@squareup.com>
Signed-off-by: John Tennant <jtennant@squareup.com>
Signed-off-by: John Tennant <jtennant@squareup.com>
Signed-off-by: John Tennant <jtennant@squareup.com>
Signed-off-by: John Tennant <jtennant@squareup.com>
Signed-off-by: John Tennant <jtennant@squareup.com>
Signed-off-by: John Tennant <jtennant@squareup.com>
@johnmatthewtennant
johnmatthewtennant force-pushed the jtennant/auto-transcribe-agent-huddles branch from 2ff30f3 to 09f8e8a Compare July 29, 2026 14:26
Signed-off-by: John Tennant <jtennant@squareup.com>
johnmatthewtennant added a commit that referenced this pull request Jul 29, 2026
@johnmatthewtennant

Copy link
Copy Markdown
Contributor Author

🤖 ## Active transcription

The transcript control uses the highlighted active state and reports pressed/on.

Active transcription

Inactive transcription

After explicit OFF, the control returns to the dark inactive state and reports pressed/off.

Inactive transcription

@johnmatthewtennant
johnmatthewtennant marked this pull request as ready for review July 29, 2026 21:41
@johnmatthewtennant johnmatthewtennant added triage-ready Appropriate for agentic review and removed triage-ready Appropriate for agentic review labels Jul 29, 2026
Preserve the PR's Tauri event listener while incorporating main's expanded E2E bridge.

Co-authored-by: npub1jh9wn95s0472h86ahapupaf7m6kx4v9sx2n0atj2hltcfer8k06s5n3pyf <95cae996907d7cab9f5dbf43c0f53edeac6ab0b032a6feae4abfd784e467b3f5@buzz.block.builderlab.xyz>

Signed-off-by: npub1jh9wn95s0472h86ahapupaf7m6kx4v9sx2n0atj2hltcfer8k06s5n3pyf <95cae996907d7cab9f5dbf43c0f53edeac6ab0b032a6feae4abfd784e467b3f5@buzz.block.builderlab.xyz>
@tlongwell-block
tlongwell-block merged commit 4632c55 into main Jul 31, 2026
25 checks passed
@tlongwell-block
tlongwell-block deleted the jtennant/auto-transcribe-agent-huddles branch July 31, 2026 11:06
tlongwell-block added a commit that referenced this pull request Jul 31, 2026
…cache

* origin/main:
  feat(desktop): auto-enable huddle transcription for agents (#3180)
  feat(agent): optional reply guard reminds a silent turn to publish (#3763)
  feat(desktop): upgrade Pocket TTS model (#3266)
  feat(desktop): delete a message by clearing its edit to empty (#3813)

Signed-off-by: npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d <011987e296fd5006292d2f930b574be47c7801048d1983c46c425d3c95f0cffd@buzz.block.builderlab.xyz>
Co-authored-by: tlongwell-block <109685178+tlongwell-block@users.noreply.github.com>
Signed-off-by: tlongwell-block <109685178+tlongwell-block@users.noreply.github.com>
tlongwell-block added a commit that referenced this pull request Jul 31, 2026
…ttings (relands #2467 + #3208) (#3910)

Relands **#2467** (extract `buzz-voice` crate) and **#3208** (Pocket
voice settings) onto main, after #3266 and #3180 merged.

## Why a fresh PR
The repo is squash-only with delete-branch-on-merge. Squashing #3266
deleted `jtennant/pocket-tts-2026-04`, which was #2467's base — GitHub
auto-closed #2467 and it cannot be reopened. Squash merges also sever
ancestry, so GitHub's natural merge-base reports phantom conflicts for
the whole remaining stack.

## Content provenance
- Byte-identical to the blessed `jt/buzz-voice-refactor` branch
(`93029c577`, tree `6729e0eff` — reviewed by Dawn (#2467) and Max
(#3208) at exact heads) **except** the three files where #3180 and #3208
genuinely interact.
- Three-file resolution (union of both sides):
- `huddle/mod.rs` — #3180's pipeline re-exports + #3208's
`agent_tts_routing` imports.
- `huddle/state.rs` — `reset_preserving_generation` preserves both
`huddle_generation` (#3180) and `tts_enabled` (#3208); test sets merged
into one `tests` module.
- `desktop/src/testing/e2eBridge.ts` — both switch arms kept; no
duplicate case labels.

## Verification at cf32dac
- `cargo test` (desktop/src-tauri, pinned 1.95.0): **2047 + 3 pass / 0
fail** (14 ignored: 8 keychain, 4 real_relay, 2 flag-gated)
- `cargo clippy --all-targets -- -D warnings`: clean; `cargo fmt
--check`: clean
- `cargo check --workspace` (root, includes new `buzz-voice` member):
clean; `cargo test -p buzz-voice`: 5/0
- `pnpm test`: **3885 / 0**; `tsc --noEmit`: clean; lint: clean

The 3180×3208 interaction resolution is getting an independent team
re-review before merge.

Buzz channel: buzz-desktop-voice `fd5fb402-b651-4238-89b1-bb3e2fa4dc96`,
thread `b4798ecc`.

Signed-off-by: npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d <011987e296fd5006292d2f930b574be47c7801048d1983c46c425d3c95f0cffd@buzz.block.builderlab.xyz>
Co-authored-by: npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d <011987e296fd5006292d2f930b574be47c7801048d1983c46c425d3c95f0cffd@buzz.block.builderlab.xyz>
wpfleger96 pushed a commit that referenced this pull request Jul 31, 2026
…evert-fix

* origin/main:
  fix(desktop): open profiles from avatars (#3751)
  refactor(voice): extract reusable Pocket primitives + Pocket voice settings (relands #2467 + #3208) (#3910)
  docs: add VISION_REMOTE_AGENTS.md (#3924)
  feat(desktop): auto-enable huddle transcription for agents (#3180)
  feat(agent): optional reply guard reminds a silent turn to publish (#3763)
  feat(desktop): upgrade Pocket TTS model (#3266)
  feat(desktop): delete a message by clearing its edit to empty (#3813)
  feat(relay): raise hosted community limit to five (#3829)
  feat(desktop): locally stored NIP-49 encrypted key backup (#2937)
  fix(catalog): update Amp tagline (#3806)
  fix(desktop): channel topic and membership metadata cleanup (#3642)
  fix(desktop): align data deletion labels (#2230)
  fix(relay): align NIP-11 max_limit with REQ ceiling (#3635)

Signed-off-by: npub1mn7jgtj4w2pd0g0zeuhxsa6jy6p0rewxz4kujt98my82ahfmp72sxjexk7 <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@buzz.block.builderlab.xyz>
wpfleger96 pushed a commit that referenced this pull request Jul 31, 2026
…chive

* origin/main: (25 commits)
  feat(desktop): import local Pocket voices (#3259)
  fix(desktop): open profiles from avatars (#3751)
  refactor(voice): extract reusable Pocket primitives + Pocket voice settings (relands #2467 + #3208) (#3910)
  docs: add VISION_REMOTE_AGENTS.md (#3924)
  feat(desktop): auto-enable huddle transcription for agents (#3180)
  feat(agent): optional reply guard reminds a silent turn to publish (#3763)
  feat(desktop): upgrade Pocket TTS model (#3266)
  feat(desktop): delete a message by clearing its edit to empty (#3813)
  feat(relay): raise hosted community limit to five (#3829)
  feat(desktop): locally stored NIP-49 encrypted key backup (#2937)
  fix(catalog): update Amp tagline (#3806)
  fix(desktop): channel topic and membership metadata cleanup (#3642)
  fix(desktop): align data deletion labels (#2230)
  fix(relay): align NIP-11 max_limit with REQ ceiling (#3635)
  fix(desktop): allow linux-only media items as dead code off-linux (#3811)
  fix(desktop): report authenticated relay recovery (#3812)
  fix(desktop): don't gate hover affordances on the hover media query (#3657)
  feat(relay): gate kind 30178 team-catalog reads behind the shared tag (#3358)
  test(desktop): click visible thread collapse guide (#3800)
  feat(desktop): raise the install ceiling and make installs observable (#3368)
  ...

Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>

# Conflicts:
#	desktop/src/testing/e2eBridge.ts
#	desktop/tests/helpers/bridge.ts
wpfleger96 pushed a commit that referenced this pull request Jul 31, 2026
…chive

* origin/main: (25 commits)
  feat(desktop): import local Pocket voices (#3259)
  fix(desktop): open profiles from avatars (#3751)
  refactor(voice): extract reusable Pocket primitives + Pocket voice settings (relands #2467 + #3208) (#3910)
  docs: add VISION_REMOTE_AGENTS.md (#3924)
  feat(desktop): auto-enable huddle transcription for agents (#3180)
  feat(agent): optional reply guard reminds a silent turn to publish (#3763)
  feat(desktop): upgrade Pocket TTS model (#3266)
  feat(desktop): delete a message by clearing its edit to empty (#3813)
  feat(relay): raise hosted community limit to five (#3829)
  feat(desktop): locally stored NIP-49 encrypted key backup (#2937)
  fix(catalog): update Amp tagline (#3806)
  fix(desktop): channel topic and membership metadata cleanup (#3642)
  fix(desktop): align data deletion labels (#2230)
  fix(relay): align NIP-11 max_limit with REQ ceiling (#3635)
  fix(desktop): allow linux-only media items as dead code off-linux (#3811)
  fix(desktop): report authenticated relay recovery (#3812)
  fix(desktop): don't gate hover affordances on the hover media query (#3657)
  feat(relay): gate kind 30178 team-catalog reads behind the shared tag (#3358)
  test(desktop): click visible thread collapse guide (#3800)
  feat(desktop): raise the install ceiling and make installs observable (#3368)
  ...

Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>

# Conflicts:
#	desktop/src/testing/e2eBridge.ts
#	desktop/tests/helpers/bridge.ts
joahg added a commit to joahg/buzz-dev-mode that referenced this pull request Jul 31, 2026
…-style

* origin/main: (22 commits)
  feat(desktop): import local Pocket voices (block#3259)
  fix(desktop): open profiles from avatars (block#3751)
  refactor(voice): extract reusable Pocket primitives + Pocket voice settings (relands block#2467 + block#3208) (block#3910)
  docs: add VISION_REMOTE_AGENTS.md (block#3924)
  feat(desktop): auto-enable huddle transcription for agents (block#3180)
  feat(agent): optional reply guard reminds a silent turn to publish (block#3763)
  feat(desktop): upgrade Pocket TTS model (block#3266)
  feat(desktop): delete a message by clearing its edit to empty (block#3813)
  feat(relay): raise hosted community limit to five (block#3829)
  feat(desktop): locally stored NIP-49 encrypted key backup (block#2937)
  fix(catalog): update Amp tagline (block#3806)
  fix(desktop): channel topic and membership metadata cleanup (block#3642)
  fix(desktop): align data deletion labels (block#2230)
  fix(relay): align NIP-11 max_limit with REQ ceiling (block#3635)
  fix(desktop): allow linux-only media items as dead code off-linux (block#3811)
  fix(desktop): report authenticated relay recovery (block#3812)
  fix(desktop): don't gate hover affordances on the hover media query (block#3657)
  feat(relay): gate kind 30178 team-catalog reads behind the shared tag (block#3358)
  test(desktop): click visible thread collapse guide (block#3800)
  feat(desktop): raise the install ceiling and make installs observable (block#3368)
  ...

Amp-Thread-ID: https://ampcode.com/threads/T-019fb8e1-6ece-72a7-8808-9b12e0f7e833
Co-authored-by: Amp <amp@ampcode.com>
Signed-off-by: Joah Gerstenberg <joah@squareup.com>

# Conflicts:
#	desktop/src-tauri/src/linux_media.rs
#	desktop/src/app/AppShell.tsx
tellaho added a commit that referenced this pull request Jul 31, 2026
…-setting

* origin/main:
  feat(desktop): import local Pocket voices (#3259)
  fix(desktop): open profiles from avatars (#3751)
  refactor(voice): extract reusable Pocket primitives + Pocket voice settings (relands #2467 + #3208) (#3910)
  docs: add VISION_REMOTE_AGENTS.md (#3924)
  feat(desktop): auto-enable huddle transcription for agents (#3180)
  feat(agent): optional reply guard reminds a silent turn to publish (#3763)
  feat(desktop): upgrade Pocket TTS model (#3266)

Signed-off-by: npub1223z34hd7vtwc6qj4s7flsxkj644nlre2nthu7lrrmkumhu3xddsrx9r6w <52a228d6edf316ec6812ac3c9fc0d696ab59fc7954d77e7be31eedcddf91335b@buzz.block.builderlab.xyz>
Co-authored-by: Taylor Ho <taylorkmho@gmail.com>
Signed-off-by: Taylor Ho <taylorkmho@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants