moq-cli ts round-trip drops multi-PPS H.264 (and DTS), breaking native broadcast passthrough

[t0ms]

moq-cli ts round-trip drops multi-PPS H.264 (and DTS), breaking native broadcast passthrough

Labels: bug, moq-mux, moq-cli

Summary

Publishing a real broadcast MPEG-TS feed and subscribing back with --format ts produces undecodable H.264: ffplay/ffprobe report non-existing PPS 0 referenced / no frame! continuously, with no playback. Root cause is that the avc3 import/export path keeps only a single SPS and a single PPS, so a source carrying multiple PPS loses all but the last-seen one. A secondary issue is that PES DTS is dropped, so B-frame content also emits DTS ... out of order.

This blocks the MVPD use case of feeding a native broadcast TS straight through a MoQ relay into an IRD unchanged.

Observed on dev.

Reproduction

# relay
cargo run --release --bin moq-relay -- demo/relay/localhost.toml

# publish a real broadcast capture (1080i H.264 + MP2)
ffmpeg -re -stream_loop -1 -i CNNiEMEA2.ts -c copy \
  -f mpegts -nostats -loglevel warning - | \
  target/release/moq-cli publish --tls-disable-verify \
    --url https://localhost:4443 --broadcast anon/my-stream ts

# subscribe -> no playback
target/release/moq-cli subscribe --tls-disable-verify \
  --url https://localhost:4443 --broadcast anon/my-stream --format ts | ffplay -
# [h264] non-existing PPS 0 referenced  /  no frame!   (forever)
# [mpegts] DTS ... out of order

Diagnosis / evidence

trace_headers on the source vs. the round-tripped output:

Source parameter sets (distinct IDs):

# PPS definitions
 533 pic_parameter_set_id = 0
1051 pic_parameter_set_id = 1
# Slice references
19082 pic_parameter_set_id = 0
  518 pic_parameter_set_id = 1

The source defines two PPS (id 0 and id 1) and ~97% of slices reference PPS 0. The subscribed output carries only the last-seen PPS (id 1), so almost every slice references a PPS that isn't present, hence non-existing PPS 0 referenced.

The mid-stream start of the capture is not the cause: it only costs the first fraction of a second (VLC tolerates it and plays the source fine). The failure is persistent because PPS 0 is missing for the whole stream.

Root causes

1. Multi-PPS collapse. Both the ingest splitter and the egress transmux hold a single parameter set each:
   • rs/moq-mux/src/codec/h264/split.rs - sps: Option<Bytes>, pps: Option<Bytes> (overwritten on each new PPS; re-injected singly on keyframes).
   • rs/moq-mux/src/codec/h264/mod.rs - Avc1 { avcc, sps: Option<Bytes>, pps: Option<Bytes> }; rebuild_avcc() builds an avcC from one SPS + one PPS.
   • H.265 (Hvc1 / Split) has the analogous single-slot limitation for VPS/SPS/PPS.
2. DTS dropped. The TS importer captures only the PES PTS (rs/moq-mux/src/container/ts/import.rs - struct Pending { pts, data, data_len }, no DTS), and the exporter hard-codes dts: None (rs/moq-mux/src/container/ts/export.rs). The hang Frame carries a single timestamp, so the PTS/DTS distinction is lost pipeline-wide (fMP4 export has the same gap: DTS = frames[0].timestamp, CTS = 0).

Proposed work

1. Multi-PPS fix (immediate blocker) - ~0.5-1 day, low risk

• Split (h264): keep an ordered set of distinct SPS/PPS NALs; re-inject all on each keyframe.
• Avc1 (h264): collect all distinct SPS/PPS; extend build_avcc to emit numOf{Sequence,Picture}ParameterSets > 1. avcc_params / parse_avcc_param_sets already loop over the counts, so TS re-injection needs no further change once the avcC carries multiples.
• Mirror in h265.
• Changes inline keyframe bytes only, not the wire/catalog format, so it is likely main-targetable and non-breaking.

2. DTS / B-frame timing

• Lighter (TS egress only, ~0.5 day, non-breaking): synthesize a monotonic DTS in the exporter from decode order (frames arrive in decode order within a group). Removes "DTS out of order".
• Proper (multi-day, breaking, dev): add a decode-timestamp / composition offset to Frame, thread it through the Legacy wire container, rs/hang, all importers/exporters, plus the js/hang mirror and doc/concept per the cross-package table.

3. Opaque MPEG-TS carriage (new passthrough mode) - ~1-2 days MVP, dev (new container/catalog = format change)

• New container/track kind the catalog can describe (e.g. Container::MpegTs / a passthrough rendition).
• Import: chunk incoming TS into groups at random-access points (video PID random_access_indicator/PUSI marks group starts for tune-in); carry packets verbatim, no codec parsing.
• Export: concatenate group bytes back to a TS stream; optionally re-stamp PCR continuity.
• Byte-faithful: preserves every PPS, DTS, SEI/captions (CEA-608/708), AFD, teletext, SCTE-35, and all audio. Cost is bounded precisely because it does no demux fidelity work.

Recommendation

• Land Improve readme #1 now (cheap, unblocks the current feed for all consumers including browser).
• Build Opening handshake failed. QUIC_TLS_CERTIFICATE_UNKNOWN #3 opaque carriage as the real primitive for MVPD native passthrough (native source -> IRD unchanged); it is the right fidelity contract and avoids whack-a-mole on captions/AFD/teletext/etc.
• Do Add server-side ABR and throttling (to test) #2 proper DTS only if B-frame-correct transcode egress (fmp4/mkv for browser/ABR) is needed; the opaque path carries original DTS in the bytes and does not need it.

The faithful codec path (#1/#2) and opaque carriage (#3) target different audiences (adaptive/browser delivery vs. native broadcast backhaul), so both are worth having.

(Written by Claude)

[coderabbitai]

🔗 Related PRs

#1767 - feat(hang): add Catalog.Producer/Consumer wrapping @moq/json [merged]
#1773 - fix(moq-relay,moq-native): stop the cert-reload busy loop, then dedupe FileWatcher [merged]
#1778 - Route moq-boy status and command tracks through moq-json [merged]
#1781 - fix(hang): omit empty HEVC constraint component in codec string [merged]
#1782 - fix(ci): unbreak the self-hosted A1 runner (libmoq clippy + ffmpeg bindgen headers) [merged]

---

📝 Issue Planner

_{Check the box below or use the @coderabbitai plan command to generate an implementation plan and prompts that you can use with your favorite coding assistant.}

• Create Plan

---

🧪 Issue enrichment is currently in open beta.

You can configure auto-planning by selecting labels in the issue_enrichment configuration.

To disable automatic issue enrichment, add the following to your .coderabbit.yaml:

issue_enrichment:
  auto_enrich:
    enabled: false

💬 Have feedback or questions? Drop into our discord!