Skip to content

feat: add native macOS audio capture - #257

Closed
marcusschiesser wants to merge 5 commits into
vercel-labs:mainfrom
marcusschiesser:codex/audio-capture-verified
Closed

feat: add native macOS audio capture#257
marcusschiesser wants to merge 5 commits into
vercel-labs:mainfrom
marcusschiesser:codex/audio-capture-verified

Conversation

@marcusschiesser

Copy link
Copy Markdown
Contributor

Closes #254.

Summary

  • add macOS 15+ system-audio, microphone, and combined audio capture to a signed 16-bit PCM WAV file
  • expose Zig effects and TypeScript commands for capture lifecycle, microphone enumeration, permission status/request, and device-change invalidation
  • add system_audio, microphone, and filesystem permission enforcement plus generated microphone/system-audio privacy purpose keys
  • retain deployability to older macOS versions and report the feature unsupported on non-AppKit hosts

Native implementation

  • use ScreenCaptureKit for system-only and combined capture, including explicit/default microphone selection
  • use AVFoundation capture sessions for microphone-only recording so screen-recording permission is unnecessary
  • timestamp-align, resample, and mix source buffers into the requested 16/24/44.1/48 kHz mono/stereo output
  • write through a sibling temporary file, finalize the WAV header, and atomically publish without overwriting an existing destination
  • preserve usable partial output on stream/device failure and finalize active capture during bounded shutdown
  • enumerate stable AVCaptureDevice.uniqueID microphone records and observe device changes only while subscribed

Runtime and API

  • add fixed capture, device, and permission event records across the platform ABI, Zig effects, TypeScript emitter/bytecode, runtime host, and session journal
  • make replay park all audio-capture operations and route recorded platform events without touching capture hardware
  • extend the null platform with deterministic devices, default changes, permissions, capture events, output collisions, and disconnect failures
  • add manifest parsing/validation/packaging for system_audio permission and microphone/system-audio purpose strings

Documentation and examples

  • add Zig and TypeScript audio-capture examples
  • add an audio-capture guide, support matrix, capability/API references, privacy guidance, sandbox entitlement guidance, and restart behavior
  • add a changelog fragment

Verification

Passing locally:

  • zig build test
  • zig build validate
  • zig build test-webview-system-link
  • npm run gate in packages/core
  • documentation typecheck/build and docs route checks
  • Zig and TypeScript audio-capture example tests
  • macOS Objective-C syntax build with macOS 11 deployment guards
  • end-to-end record/replay regression covering start, stop, device enumeration, access status, device-change invalidation, identical replay state, and zero replay-side platform capture calls

scripts/gate.sh fast was also exercised. The touched suites pass; the remaining local limitations were:

  • the pre-existing Ghostty-backed terminal/workbench examples could not download their pinned archive because GitHub DNS resolution timed out
  • the CEF link leg was skipped because no local CEF bundle is installed
  • the unrelated toggle-big-view render benchmark measured about 10.18 ms against its 9 ms budget; all other benchmark scenarios passed

Manual microphone and Screen Recording TCC flows still need to be exercised on a macOS 15+ machine before moving this PR out of draft.

@vercel

vercel Bot commented Aug 1, 2026

Copy link
Copy Markdown

@marcusschiesser is attempting to deploy a commit to the Vercel Labs Team on Vercel.

A member of the Team first needs to authorize it.

@marcusschiesser

Copy link
Copy Markdown
Contributor Author

Superseded by #264, which implements the replacement reliable paired-stream design from #262. File/container writing is now separated into #263.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Design: add macOS system and microphone audio capture

1 participant