spec: fix segfault error on long prompts for eagle3

[ruixiang63]

Overview

Fix #24637

Root cause:

Eagle3 speculative decoding crashes with a segmentation on long prompts. The draft decoder sizes its input-embeddings batch with the wrong embedding dimension, producing an out-of-bounds read that only manifests once the prompt is long enough for the copy offset to cross the allocated buffer.

Printing n_embd_inp / n_embd / n_embd_out shows a draft-only mismatch (Gemma4 26B-A4B):

	n_embd_inp	n_embd	n_embd_out
target	2816	2816	2816
draft (eagle3)	8448	2816	2816

Eagle3 sets n_embd_inp = target_layers * target_hidden (8448) for the draft encoder, but the draft decoder consumes g_embeddings of width n_embd (2816). decode() sizes the embd batch with n_embd_inp(), so ubatch_add() reads 8448 floats/row from a 2816-wide buffer → out-of-bounds. The offset scales with prompt length, so short prompts work and long prompts segfault.

A one-liner works arch == EAGLE3 ? n_embd : n_embd_inp() (see https://github.com/ruixiang63/llama.cpp/blob/7dd72f620185f74e75eaf4d265783207eb7708d2/src/llama-context.cpp#L1695), but it adds an arch-specific branch to the generic decode path.

Instead, this PR stops overloading n_embd_inp:

• add n_embd_inp_enc(): encoder input dim, falls back to n_embd_inp()
• eagle3 stores target_layers * target_hidden in n_embd_inp_enc_impl, so n_embd_inp() == n_embd again
• encode() / draft encoder use n_embd_inp_enc(); decode() unchanged
  No-op for non-eagle3 models: n_embd_inp_enc() returns n_embd_inp() whenever the field is 0.

Tested on gemma-4-26B target + eagle3 draft, ~30K-token prompt: segfaults before, completes after.

Note

During testing, I found low long-context acceptance is a draft-model limitation, not an implementation bug.

This EAGLE3 draft (gemma-4-26B speculator) accepts well on short prompts but collapses on long ones (~30K): per-position 0.14, 0, 0, 0, mean length ~1.15.

Cross-checked against the vLLM reference (same draft + target + prompt) — it reports the same: Per-position acceptance rate: 0.139, 0.013, 0.000, 0.000, mean 1.15.

python -m vllm.entrypoints.openai.api_server \
  --model gemma-4-26B-A4B-it \
  --served-model-name gemma4 \
  --host 0.0.0.0 \
  --port 8000 \
  --max-model-len 32768 \
  --gpu-memory-utilization 0.6 \
  --reasoning-parser gemma4 \
  --speculative-config '{"model":"RedHatAI/gemma-4-26B-A4B-it-speculator.eagle3","draft_tensor_parallel_size":1,"num_speculative_tokens":4,"method":"eagle3"}' \
  --enforce-eager

Since the reference engine reproduces it, this is a draft-model quality issue. Looking deeper: its rope_theta = 10000 short-range RoPE doesn't extrapolate to long context.

Requirements

• I have read and agree with the contributing guidelines
• AI usage disclosure: Yes, paired with cursor for debugging and design