spec: support spec-draft-p-min in DFlash#25246
Merged
Merged
Conversation
Member
Author
|
@ggml-org/maintainers Can I get a quick review for this simple change? Thanks! |
pwilkin
approved these changes
Jul 3, 2026
gaugarg-nv
approved these changes
Jul 3, 2026
pwilkin
approved these changes
Jul 3, 2026
LyndonBlack
added a commit
to LyndonBlack/llama.cpp-Ternary-1.58Bit-and-TurboQuant
that referenced
this pull request
Jul 5, 2026
Sync with upstream (44 commits): - DFlash speculative decoding support (ggml-org#22105, ggml-org#25110, ggml-org#25246) - DeepSeek V4 model support (ggml-org#24162) - MoE topk fusion for 288 experts (ggml-org#25267) - Qwen3Next model fixes (ggml-org#25141) - CUDA FA optimizations + Gemma E4B MTP fix - NVFP4 support for WebGPU, AVX2 nvfp4 dot - llama_model_ftype_name() API - Various UI, server, and build fixes Resolved conflicts: - arch-fallback.h: kept Q2_0 vec_dot generic defines - llama-kv-cache-iswa.{h,cpp}: merged upstream 2-ctor refactor with turboquant layer_type_k_cb - llama-model-loader.cpp: adopted upstream llama_ftype_name() with Q2_0 entry preserved Assisted-by: Hermes Agent
adrianhoehne
pushed a commit
to adrianhoehne/llama.cpp
that referenced
this pull request
Jul 5, 2026
* spec: support spec-draft-p-min in DFlash * dflash: add n_min guard * dflash: guard both n_min and n_max
TheTom
pushed a commit
to TheTom/llama-cpp-turboquant
that referenced
this pull request
Jul 9, 2026
* spec: support spec-draft-p-min in DFlash * dflash: add n_min guard * dflash: guard both n_min and n_max (cherry picked from commit 152d337)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Overview
Allow user to set
--spec-draft-p-minin DFlash.I thought DFlash is a diffusion-based draft model rather than an autoregressive one, so I initially assumed that
spec-draft-p-minwasn't very important because truncating the draft doesn't reduce the draft generation time.However, it can still be useful for reducing the target model's verification time by truncating the draft token once a token's confidence falls below the threshold specified by
--spec-draft-p-min.Therefore, in some use cases, tuning this parameter can improve overall performance.
Additional information
Requirements