refactor(models): share the short-conv decode helper and audit L=1 conv dispatch across SSM/hybrid decode paths

[inureyes]

Summary

Follow-up to #748 / PR #751 (3fa2789). That fix found MLX 0.32.1's CUDA backend dispatches a single-output-position (L=1) bf16 depthwise conv1d to cuDNN's generic per-channel engine (convolve_common_engine_float_NHWC, one launch per channel), which regressed LFM2 decode ~10x, and replaced the L=1 step with a broadcast multiply plus axis sum. This PR does two things: (1) lifts the #751 decode helpers into a shared module so any family can adopt them in one line, and (2) measures every other decode-path depthwise-conv family on GB10 to see whether the same slow dispatch recurs.

It does not. Every SSM / hybrid family already runs its L=1 decode conv on the fast conv1d_c1_k1_nhwc cuDNN engine, so none is adapted. The result is a pure refactor plus a documented, measurement-driven negative result.

What changed

• src/models/conv_decode.rs (new): shared build_conv_decode_weight + short_conv_decode_step, lifted verbatim from lfm2.rs. src/models/lfm2.rs re-exports them at the original path (pub(crate) use), so ShortConv and the existing 11 lfm2 tests are unchanged. Pure move, behavior identical (lfm2-350m-8bit greedy output byte-identical: "Hello! I'm doing well, thanks. How about you?", 13 tokens, and decode still elementwise at 393-ish tok/s).
• src/models/conv_decode_tests.rs (new): generalized channel- and tap-asymmetric parity tests (f32 + bf16, kernel widths 3 and 4, plus a silu(bias + conv) end-to-end variant) pinning the elementwise decode step against conv1d without a checkpoint.
• src/models/mod.rs: wire conv_decode under the shared modules and conv_decode_tests under the test modules.
• docs/benchmark_results/model_tests_gb10.md: record the audit outcome, annotate the mamba2-130m row, and correct the PR fix(cuda): restore lfm2-350m-8bit decode via elementwise short conv #751 hypothesis on the gemma-4 / glm4-flash drops.

Method

Each family with a local checkpoint was profiled under nsys on a warm GB10 CUDA decode (mlxcel generate ... -n 24 --profile, MLX_USE_CUDA_GRAPHS=0 so the cuDNN engine choice is captured eagerly and the histogram is complete), then nsys stats --report cuda_gpu_kern_sum was grepped for conv kernels. The cuDNN engine choice is keyed on shape/dtype/alignment/groups and is independent of MLX graph capture, so graphs-off is representative of production.

The discriminator is the launch count, not just the kernel name. The fast conv1d_c1_k1_nhwc engine launches one kernel per conv op; the slow convolve_common_engine launches one per channel (LFM2's 350M hit 286,720 launches, ~1024 per op). Every family measures at (conv layers x decode tokens) launches, i.e. one per op, which is direct positive proof of the fast engine rather than mere absence of the slow name.

Per-family verdict

No family was CONFIRMED, so there are no before/after tok/s (nothing was adapted). Kernel counts are from the warm 24-token decode.

Family	Checkpoint	Act. dtype	Conv engine	conv1d_c1_k1_nhwc launches	convolve_common_engine	conv % of decode	Verdict
mamba2	mamba2-130m	bf16	fast	576 (24 layers x 24 tok)	0	1.6%	REFUTED
mamba2	mamba2-1.3b-4bit	fp16	fast	1152	0	1.7%	REFUTED
mamba	falcon-mamba-7b-4bit	fp16	fast	256 (early EOS, 2 tok)	0	0.9%	REFUTED
falcon-h1	falcon-h1-tiny-90m-instruct-4bit	bf16	fast	576 (24 x 24)	0	3.0%	REFUTED
granite-4.0-h	granite-4.0-h-350m-4bit	bf16	fast	672	0	2.1%	REFUTED
jamba	jamba-v0.1-4bit	bf16	fast	624	0	1.0%	REFUTED
plamo2	plamo-2-1b	f32	fast	192	0	0.2%	REFUTED
qwen3.5 / gated_delta	qwen3.5-0.8b-4bit	bf16	fast	432	0	1.9%	REFUTED
nemotron-h	nemotron-h-30b-4bit	bf16	fast	552 (23 layers x 24 tok)	0	0.7%	REFUTED
kimi_linear (ShortConv1d)	kimi-vl-a3b-thinking-4bit	n/a	n/a	0 (no conv runs)	0	n/a	NOT MEASURED
recurrent_gemma	none on disk	n/a	n/a	n/a	n/a	n/a	NOT MEASURED
qwen3-next	none on disk	n/a	n/a	n/a	n/a	n/a	NOT MEASURED

Notes on the NOT MEASURED rows:

• kimi_linear: the listed kimi-vl-a3b-thinking-4bit is a deepseek_v3 text backbone (MLA attention, no conv); it never exercises kimi_linear.rs's ShortConv1d, so nsys shows no conv kernel at all. The actual Kimi-Linear architecture has no local checkpoint. Its ShortConv1d (conv1d -> silu, no bias) would fit the shared helper if a checkpoint appears.
• recurrent_gemma: no local checkpoint, and its conv is structurally different (it transposes to [B, C, L], convolves, transposes back), so it would need a small local adaptation rather than the shared helper. Documented, not adapted.
• qwen3-next: no local checkpoint. It shares the gated-delta conv that qwen3.5-0.8b-4bit exercises and which is REFUTED above.

Why LFM2 regressed but these do not

LFM2's slow dispatch is specific to its conv_L_cache = 3 (kernel width 3) at hidden 1024. cuDNN's engine heuristic selects the fast NHWC engine for the SSM families' conv_kernel = 4 shapes and the slow generic per-channel engine only for LFM2's shape. The conv share of decode is 0.2-3.0% everywhere here, so even if a shape did tip over, the ceiling is small; adapting a refuted family would add the elementwise path for no benefit (and could be slower than the fast cuDNN kernel), so none is changed.

PR #751 hypothesis correction (on the record)

PR #751's body speculated that the gemma-4-26b-a4b-it(-qat) (-10 to -14%) and glm4-flash-4bit (-14%) sweep drops were the same conv-dispatch pattern. Code inspection refutes that: neither is a conv architecture (gemma4 MoE and glm4_moe_lite; neither file calls conv1d), so their decode conv cannot fall into the grouped-conv engine. Those drops need separate attribution and are out of scope here. Similarly, mamba2-130m's -10.4% sweep delta is not conv dispatch (its conv is on the fast engine at 1.6% of decode); it is environmental sweep variance, as the issue anticipated.

Test plan

• cargo test --lib --features cuda models::conv_decode (5 passed: f32/bf16 kernel-3 and kernel-4 parity + silu/bias variant)
• cargo test --lib --features cuda models::lfm2 (11 passed, unmodified)
• cargo clippy -p mlxcel --lib --tests --features cuda (clean; only pre-existing mlxcel-core C++ build notes)
• cargo fmt (no changes)
• cargo build --release --features cuda then nsys audit of all 10 candidate checkpoints on GB10
• lfm2-350m-8bit greedy output byte-identical after the refactor

Closes #752