Hy3 (295B): CUDA backend support, GQA attention path#535

Open

giannisanni wants to merge 7 commits into

antirez:glm5.2from

giannisanni:hy3-cuda-pr

Commits on Jul 7, 2026

GLM 5.2: CUDA backend support
giannisanni
committed

Commits on Jul 10, 2026

Hy3 (hy_v3) port foundation: model recognition + GQA attention kernels
giannisanni
committed
Hy3 inference path: forward_token, generate_hy3_cuda_argmax, dispatch + gate (untested)
giannisanni
committed
Hy3: bind exp_probs_b without .bias suffix (matches real hy-v3 GGUF)
giannisanni
committed
Hy3: resolve bos/eos by metadata token ids, not DeepSeek literals
giannisanni
committed
GQA attention: single-pass online softmax, kill the pass-2 score recompute
giannisanni
committed
Hy3: interactive chat mode with KV retained across turns (GLM parity)
giannisanni
committed