fix(server): prevent GGML_ABORT when prompt cache pos_min == -1 for non-standard attention architectures#1
Merged
nisparks merged 1 commit intoMay 6, 2026
Conversation
When non-standard attention architectures (e.g. DeepSeek V4 Flash CSA+HCA) are used, llama_memory_seq_pos_min() may return -1 even with n_past > 0. The custom KV cache layout is incompatible with standard prompt cache restoration, causing a hard crash. Replace GGML_ABORT with graceful fallback: set n_past=0 and pos_next=0 to force full prompt re-evaluation, same as the SWA/hybrid memory path. Verified: 60+ concurrent requests on 8xA100, zero crashes. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
xczhanjun
added a commit
to tokencube/llama.cpp
that referenced
this pull request
May 2, 2026
…k V4 Rebrands user-visible llama.cpp → zllama.io across WebUI constants, page titles, MCP identifiers, localStorage keys, and HTTP Server header. Adds APP_NAME import to chat page. Applies prompt cache pos_min==-1 graceful fallback patch (nisparks#1) to prevent GGML_ABORT. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
nisparks
approved these changes
May 6, 2026
nisparks
added a commit
that referenced
this pull request
May 6, 2026
PR #1 (tokencube) added a smaller and better fix at the abort site itself: when llama_memory_seq_pos_min returns -1 mid-request, fall back to full prompt re-evaluation instead of aborting. With that fallback in place the heavy-handed startup-time cache disable is no longer needed -- the cache stays useful for full-prefix matches even on non-PART models, and the rare prefix-mismatch path now gracefully degrades. Reverts the Force the prompt cache off in that case to avoid a confusing GGML_ABORT block from 70689ee.
Author
|
Thanks for your great work.
…On Thu, May 7, 2026 at 12:26 AM Nicholas Sparks ***@***.***> wrote:
Merged #1 <#1> into
wip/deepseek-v4-support.
—
Reply to this email directly, view it on GitHub
<#1 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAEV5OTZFXCXSKP2Z6U57U34ZNRSPAVCNFSM6AAAAACYOAYTXCVHI2DSMVQWIX3LMV45UABCJFZXG5LFIV3GK3TUJZXXI2LGNFRWC5DJN5XDWMRVGIZDOMZYGUZDANA>
.
You are receiving this because you authored the thread.Message ID:
***@***.***>
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
When using non-standard attention architectures (e.g. DeepSeek V4 Flash with CSA+HCA),
llama_memory_seq_pos_min()may return -1 even whenn_past > 0. The custom KV cache layout is incompatible with standard prompt cache restoration, causing a hard crash:This happens consistently with
--parallel > 1 --cont-batchingon DeepSeek V4 Flash, typically on the 2nd-4th concurrent request.Fix
Replace
GGML_ABORTwith graceful fallback: setn_past = 0andpos_next = 0to force full prompt re-evaluation. This mirrors the existingdo_resetpath for SWA/hybrid/recurrent memory models. 4-line change.Testing
ab -n 20 -c 4--parallel 4 --cont-batching --ctx-size 32768Compatibility
Backward-compatible. Standard architectures unaffected (pos_min returns valid value, fallback never triggers). Non-standard architectures gracefully degrade instead of crashing.
Upstream issue reference: ggml-org#13833 (comment)