fix(moq-mux): reject frames before the first keyframe instead of dropping

[kixelated]

Summary

A MoQ group must start with a keyframe. The container Producer papered over this with a with_lenient_start opt-in that silently dropped non-keyframes arriving before the first group. That let a malformed stream produce an undecodable group with no signal, and the leniency was load-bearing in ways that weren't obvious at the call sites.

This removes with_lenient_start and makes the invariant explicit, with a single legitimate exception (MPEG-TS, which can join mid-GOP).

What changed

• Producer owns and enforces the invariant (producer.rs): a non-keyframe with no open group is rejected with a new typed Error::MissingKeyframe. write now returns crate::Error directly (the bound crate::Error: From<C::Error> holds for every container error, incl. fmp4::Error via the Cmaf variant), so importers just ?-propagate the error rather than re-deriving the decision. Removed with_lenient_start.
• New typed error (error.rs): Error::MissingKeyframe plus Error::is_missing_keyframe(&anyhow::Error) to detect it through an anyhow chain.
• Codec importers (h264, h265) reset per-frame state before writing, so a propagated MissingKeyframe leaves them ready for the next access unit. The h264 avc3 path additionally errors on a keyframe it cannot configure (no inline SPS, no avcC seed) — the genuinely undecodable case, kept distinct from MissingKeyframe so it is never swallowed.
• TS is the sole exception (ts/import.rs): a transport stream legitimately joins mid-GOP, so Stream::write ignores MissingKeyframe. This is the single choke point, so it also covers the post-seek discontinuity case.
• FLV propagates it (flv/import.rs): a valid FLV is expected to open on a keyframe, so a leading delta surfaces as a malformed-stream error to the caller.

Why

The old behavior silently swallowed a real protocol violation. Making it a typed error that the Producer returns by default — with mid-stream tolerance opt-in at exactly one importer — means a group can never start without a keyframe, the one format where mid-GOP join is normal (TS) is explicitly the special case, and the keyframe decision lives in one place instead of being re-derived at every call site.

Public API note

Removing pub fn with_lenient_start from moq-mux's container Producer is technically a breaking change, but the only callers were internal to the crate. Producer::write/finish/seek/finish_group now return crate::Error instead of the (always-crate::Error in practice) associated C::Error. Error::MissingKeyframe (a #[non_exhaustive] enum) and Error::is_missing_keyframe are additive. moq-mux isn't in the explicit breaking-change list, so this targets main — happy to retarget dev if reviewers prefer.

Test plan

• cargo test -p moq-mux — 236 passed, incl. survives_midstream_join and kyrion_dirtystart_extracts_real_cues
• New: keyframe_without_sps_errors, delta_before_init_returns_missing_keyframe; updated first_frame_must_be_keyframe to expect MissingKeyframe
• cargo clippy -p moq-mux --all-targets clean
• cargo fmt --check clean (via nix toolchain)
• Dependents compile (hang, moq-cli, moq-native)

No cross-package sync needed: the JS side (js/hang) has no equivalent leniency concept.

(Written by Claude)

[coderabbitai]

Walkthrough

The pull request removes the with_lenient_start() builder method and the associated lenient_start field from Producer<C>, replacing the silent delta-drop behavior with an explicit Error::MissingKeyframe variant and a new has_group() -> bool query method. Every importer (H.264 avc3 and avc1 paths, H.265, FLV) removes its .with_lenient_start() call and adds a has_group() guard that returns MissingKeyframe when a delta arrives before any group is open. The H.264 avc3 path also hard-fails when a keyframe arrives before SPS-derived configuration. The TS importer captures the decode result and suppresses MissingKeyframe errors, returning Ok(()) for late-joining streams. Two tests validate the avc3 IDR-before-SPS and delta-before-keyframe error paths.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title 'reject frames before the first keyframe instead of dropping' accurately summarizes the main change—converting from silent frame dropping to explicit rejection of non-keyframes before the first keyframe.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Description check	✅ Passed	The pull request description clearly explains the changes, rationale, and implementation details aligned with the changeset modifications.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

✨ Simplify code

• Create PR with simplified code
• Commit simplified code in branch claude/romantic-lovelace-133ded

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[kixelated]

@arielmol I missed with_lenient_start in the original PR.

I'll fix this on the dev branch, but IMO we should error on invalid input instead of skipping. The TS importer can be a special case that allows joining mid-stream, but like we should tell the user they doing stuff wrong instead of silently accepting it IMO.

I think on the dev branch I'll add a MissingKeyframe error that the TS importer will ignore.