arch : add missing tensor name for Qwen 3.5

[ownia]

Overview

Fix #21115

Additional information

Add missing tensor names for Qwen 3.5 to avoid the following warnings when running llama-quantize Qwen3.5-4B.gguf Qwen3.5-4B-Q4.gguf Q4_0:

...
llama_model_loader: - kv  33:                         tokenizer.ggml.pre str              = qwen35
llama_model_loader: - kv  34:                      tokenizer.ggml.tokens arr[str,248320]  = ["!", "\"", "#", "$", "%", "&", "'", ...
llama_model_loader: - kv  35:                  tokenizer.ggml.token_type arr[i32,248320]  = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
llama_model_loader: - kv  36:                      tokenizer.ggml.merges arr[str,247587]  = ["Ġ Ġ", "ĠĠ ĠĠ", "i n", "Ġ t",...
llama_model_loader: - kv  37:                tokenizer.ggml.eos_token_id u32              = 248046
llama_model_loader: - kv  38:            tokenizer.ggml.padding_token_id u32              = 248044
llama_model_loader: - kv  39:               tokenizer.ggml.add_bos_token bool             = false
llama_model_loader: - kv  40:                    tokenizer.chat_template str              = {%- set image_count = namespace(value...
llama_model_loader: - type  f32:  177 tensors
                                                                                                                                                                  llama_model_loader: - type bf16:  249 tensors
str: cannot properly format tensor name position_embd with suffix=weight bid=-1 xid=-1
str: cannot properly format tensor name token_types with suffix=weight bid=-1 xid=-1
str: cannot properly format tensor name position_embd with suffix=weight bid=-1 xid=-1
str: cannot properly format tensor name token_types with suffix=weight bid=-1 xid=-1
str: cannot properly format tensor name position_embd with suffix=weight bid=-1 xid=-1
str: cannot properly format tensor name token_types with suffix=weight bid=-1 xid=-1
str: cannot properly format tensor name position_embd with suffix=weight bid=-1 xid=-1
str: cannot properly format tensor name token_types with suffix=weight bid=-1 xid=-1
str: cannot properly format tensor name position_embd with suffix=weight bid=-1 xid=-1
str: cannot properly format tensor name token_types with suffix=weight bid=-1 xid=-1
str: cannot properly format tensor name position_embd with suffix=weight bid=-1 xid=-1
str: cannot properly format tensor name token_types with suffix=weight bid=-1 xid=-1
str: cannot properly format tensor name position_embd with suffix=weight bid=-1 xid=-1
str: cannot properly format tensor name token_types with suffix=weight bid=-1 xid=-1
str: cannot properly format tensor name position_embd with suffix=weight bid=-1 xid=-1
str: cannot properly format tensor name token_types with suffix=weight bid=-1 xid=-1
...

Requirements

• I have read and agree with the contributing guidelines
• AI usage disclosure: NO

[CISC]

This is not the correct fix (Qwen 3.5 does not have these tensors), the log is erroneous and affects all models.

cc/ @JohannesGaessler

[ownia]

Oh sorry indeed there are not, double confirmed with gguf_dump. This is a common issue that affects all models.

[JohannesGaessler]

I mean, for now we can also just disable the warning again. The problem is that there is a mismatch between the tensors that a model should have according to llm_get_tensor_names and the tensors that are actually being created which will result in tensors with an incorrectly formatted name. But I don't know why we have llm_get_tensor_names in the first place, it seems to me that it should just be removed and the tensor names just always formatted based on what arguments are being supplied to str().

[CISC]

I guess the real issue is that quantize does this with no check if arch actually has those:

llama.cpp/src/llama-quant.cpp

Lines 317 to 319 in fcc2d59

	// do not quantize positional embeddings and token types (BERT)
	quantize &= name != LLM_TN(arch)(LLM_TENSOR_POS_EMBD, "weight");
	quantize &= name != LLM_TN(arch)(LLM_TENSOR_TOKEN_TYPES, "weight");

[CISC]

But I don't know why we have llm_get_tensor_names in the first place, it seems to me that it should just be removed and the tensor names just always formatted based on what arguments are being supplied to str().

I suppose it was meant as a safeguard, but it may be superfluous, we should probably look into if it can just be removed.

[ownia]

I guess the real issue is that quantize does this with no check if arch actually has those:

llama.cpp/src/llama-quant.cpp

Lines 317 to 319 in fcc2d59

	// do not quantize positional embeddings and token types (BERT)
	quantize &= name != LLM_TN(arch)(LLM_TENSOR_POS_EMBD, "weight");
	quantize &= name != LLM_TN(arch)(LLM_TENSOR_TOKEN_TYPES, "weight");

Yes, I was just about to post this snippet. If quantize needs to disable the warning and not perform additional checks, there is a way we can revert this to:

    quantize &= name.find("position_embd.weight") == std::string::npos;
    quantize &= name.find("token_types.weight") == std::string::npos;

[CISC]

Yes, I was just about to post this snippet. If quantize needs to disable the warning and not perform additional checks, there is a way we can revert this to:

    quantize &= name.find("position_embd.weight") == std::string::npos;
    quantize &= name.find("token_types.weight") == std::string::npos;

Yes, honestly don't know why it didn't do that in the first place.

[CISC]

@ownia gentle ping, do you want to update this PR?

[ownia]

Hi @CISC , I have created another pull request (#21544) to extand this discussion. We can then close this PR.