Summary
Add DeepSeek-V4 support. mlxcel has DeepSeek V2 / V3 / V3.2 but not V4.
Scope correction (2026-07)
An initial port attempt (PR #592) assumed "V4 = a V3 backbone + HiSA + a shared-expert swiglu_limit clamp" and implemented DeepSeekV4Model as a thin wrapper over DeepSeekV3Model. That premise is incorrect. Verified against the in-tree reference references/mlx-vlm/mlx_vlm/models/deepseek_v4/ and the real DeepSeek-V4-Flash checkpoint, DeepSeek-V4 is a genuinely novel architecture, not a V3 variant. A V3 wrapper does not load or run a real V4 checkpoint, so PR #592 was closed without merging and this issue is re-scoped to a faithful port of the real architecture.
Real V4 architecture (port target)
Port references/mlx-vlm/mlx_vlm/models/deepseek_v4/ faithfully. The distinguishing components are:
- HyperConnections in place of plain residual connections.
- A pooled-KV
Compressor (not V3-style MLA).
- HiSA hierarchical sparse attention via an
Indexer.
- Hash-routed MoE with
sqrtsoftplus gating (not V3 group-limited softmax routing).
- A
MultiLinear output projection.
Scope
- Add
ModelType::DeepSeekV4 + a detection.rs arm for deepseek_v4 backed by a real V4 backbone (not a V3 alias).
- Implement the components above as net-new modules, reusing V3 pieces only where genuinely shared.
- Register across the arch/metadata tables and
docs/supported-models.md; add unit + reference-parity tests.
- Validate on the real
DeepSeek-V4-Flash checkpoint (available locally at /home/inureyes/models/deepseek-v4-flash-4bit).
Effort
High: net-new, multi-component architecture. Consider splitting HyperConnections, the pooled-KV Compressor, the HiSA Indexer, and hash-routed MoE into dedicated sub-issues.
Notes
The rejected V3-wrapper attempt is preserved in the closed PR #592 for reference only.
Summary
Add DeepSeek-V4 support. mlxcel has DeepSeek V2 / V3 / V3.2 but not V4.
Scope correction (2026-07)
An initial port attempt (PR #592) assumed "V4 = a V3 backbone + HiSA + a shared-expert
swiglu_limitclamp" and implementedDeepSeekV4Modelas a thin wrapper overDeepSeekV3Model. That premise is incorrect. Verified against the in-tree referencereferences/mlx-vlm/mlx_vlm/models/deepseek_v4/and the realDeepSeek-V4-Flashcheckpoint, DeepSeek-V4 is a genuinely novel architecture, not a V3 variant. A V3 wrapper does not load or run a real V4 checkpoint, so PR #592 was closed without merging and this issue is re-scoped to a faithful port of the real architecture.Real V4 architecture (port target)
Port
references/mlx-vlm/mlx_vlm/models/deepseek_v4/faithfully. The distinguishing components are:Compressor(not V3-style MLA).Indexer.sqrtsoftplusgating (not V3 group-limited softmax routing).MultiLinearoutput projection.Scope
ModelType::DeepSeekV4+ adetection.rsarm fordeepseek_v4backed by a real V4 backbone (not a V3 alias).docs/supported-models.md; add unit + reference-parity tests.DeepSeek-V4-Flashcheckpoint (available locally at/home/inureyes/models/deepseek-v4-flash-4bit).Effort
High: net-new, multi-component architecture. Consider splitting HyperConnections, the pooled-KV Compressor, the HiSA Indexer, and hash-routed MoE into dedicated sub-issues.
Notes
The rejected V3-wrapper attempt is preserved in the closed PR #592 for reference only.