llama : make loras compatible with repacking#12593
Merged
Merged
Conversation
Member
Author
|
@giladgd @amakropoulos Could you test if this change fixes the LoRA usage with |
Contributor
|
@ggerganov yes that fixed it! I get these warnings but otherwise works as expected: |
Member
Author
|
Yes, the warnings are expected. |
ggml-ci
Contributor
|
@ggerganov I've run some tests and this seems to solve the issue. |
slaren
approved these changes
Mar 26, 2025
slaren
left a comment
slaren
left a comment
Member
There was a problem hiding this comment.
This should work for now. Ideally we would use the same process as the model weights to determine the buffer type. I think this could be done by extracting to a function the relevant parts of the create_tensor lambda in llama_model::load_tensors, and keeping a mapping of each weight to its LLM_TN_IMPL object in llama_model.
Contributor
|
Thank you for the quick fix! |
Seunghhon
pushed a commit
to Seunghhon/llama.cpp
that referenced
this pull request
Apr 26, 2026
* llama : make loras compatible with repacking ggml-ci * cont : simplify ggml-ci * cont : add TODO [no ci]
ljubomirj
pushed a commit
to ljubomirj/llama.cpp
that referenced
this pull request
May 6, 2026
* llama : make loras compatible with repacking ggml-ci * cont : simplify ggml-ci * cont : add TODO [no ci]
my-other-github-account
pushed a commit
to my-other-github-account/llama.cpp
that referenced
this pull request
May 15, 2026
* llama : make loras compatible with repacking ggml-ci * cont : simplify ggml-ci * cont : add TODO [no ci]
my-other-github-account
pushed a commit
to my-other-github-account/llama.cpp
that referenced
this pull request
May 15, 2026
* llama : make loras compatible with repacking ggml-ci * cont : simplify ggml-ci * cont : add TODO [no ci]
phibya
pushed a commit
to ziee-ai/llama.cpp
that referenced
this pull request
May 29, 2026
* llama : make loras compatible with repacking ggml-ci * cont : simplify ggml-ci * cont : add TODO [no ci]
AlexiAlp
pushed a commit
to minghaop/llama.cpp
that referenced
this pull request
Jun 2, 2026
* llama : make loras compatible with repacking ggml-ci * cont : simplify ggml-ci * cont : add TODO [no ci]
AlexiAlp
pushed a commit
to minghaop/llama.cpp
that referenced
this pull request
Jun 2, 2026
* llama : make loras compatible with repacking ggml-ci * cont : simplify ggml-ci * cont : add TODO [no ci]
fukuro-kun
pushed a commit
to fukuro-kun/fukuro-llama-cpp-turboquant
that referenced
this pull request
Jul 5, 2026
* llama : make loras compatible with repacking ggml-ci * cont : simplify ggml-ci * cont : add TODO [no ci]
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
fix #12587
ref #12181 (comment)
When a LoRA adapter requires to be loaded to an extra buffer type (such as repacked bufts), load in to the default CPU buffer type instead.