Skip to content

fix(server): make audio synthesis robust to MLX FFI exceptions (cxx std::terminate bypasses the panic backstop) #382

Description

@inureyes

Problem / Background

The audio synthesis and transcription forward paths (src/server/kokoro_tts.rs, src/models/kokoro/*, src/server/whisper_stt.rs) build MLX graphs through mlxcel-core ops that are declared in the cxx bridge (src/lib/mlxcel-core/src/lib.rs) as -> UniquePtr<MlxArray>, i.e. NOT Result. Only a few boundaries (for example try_eval) return Result.

When the underlying MLX C++ throws (shape mismatch, allocation failure, etc.) inside a non-Result cxx function, cxx hard-aborts the process via std::terminate. This is NOT a Rust panic, so it is NOT intercepted by the catch_unwind backstop in src/server/audio_worker.rs (run_guarded), and it cannot be caught regardless of the release panic setting (abort or unwind).

Consequence: an MLX FFI exception on the audio forward path aborts the whole server process. This is the dominant residual fault vector for the audio path and is the one class that the Rust-panic-class remediations cannot cover. The existing input guards (4096-char input cap, Kokoro-vocab-restricted g2p output, 510-token truncation, finite/positive speed, per-token frame counts clamped to [1,100], whitelisted voice names) make it largely unreachable today, but it remains the gap.

Proposed Solution

Make the MLX ops used on the audio forward path fallible at the FFI boundary so a C++ exception becomes a recoverable Err instead of a std::terminate. Options to evaluate:

  • (a) Add Result-returning variants of the specific ops used on the audio forward path (cxx converts the thrown C++ exception into Err), and thread them through the Kokoro/Whisper forward code. Scoped to the audio path.
  • (b) A broader mlxcel-core policy of Result-returning graph ops. Larger, cross-cutting, affects every model. Likely out of scope for a first pass.

Feasibility and the exact op surface to convert need assessment. This is a mlxcel-core FFI change, intentionally separated from the panic-safety issue #375 (which addresses only Rust-panic-class containment).

Acceptance Criteria

  • An MLX C++ exception raised on the audio synthesis or transcription forward path results in a structured per-request error, not a process abort, in a release build.
  • A regression test (or documented manual verification) demonstrates the behavior.
  • No measurable regression to the audio decode/throughput path from the added fallibility.

Reference

Surfaced during the analysis of #375 (panic-safety for audio synthesis). Relates to the #374 catch_unwind backstop and the op declarations in src/lib/mlxcel-core/src/lib.rs. See src/server/audio_worker.rs (run_guarded).

Metadata

Metadata

Assignees

Labels

area:inferenceGeneration, sampling, decoding (incl. speculative, DRY)area:modelsModel architectures, weights, loading, metadatapriority:lowLow prioritystatus:doneCompletedtype:securitySecurity vulnerability or fix

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions