[Bug] VideoQnA UT test fail.

[ZePan110]

Priority

P1-Stopper

OS type

Ubuntu

Hardware type

Xeon-GNR

Installation method

• Pull docker images from hub.docker.com
• Build docker images from source
• Other
• N/A

Deploy method

• Docker
• Docker Compose
• Kubernetes Helm Charts
• Kubernetes GMC
• Other
• N/A

Running nodes

Single Node

What's the version?

afb46bd

Description

https://github.com/opea-project/GenAIExamples/actions/runs/15189596329/job/42752387727

Reproduce steps

cd VideoQnA/tests
bash test_compose_on_xeon.sh

Raw log

Attachments

No response

[xiguiw]

Priority

P1-Stopper

OS type

Ubuntu

Hardware type

Xeon-GNR

Installation method

• Pull docker images from hub.docker.com[x] Build docker images from source[ ] Other[ ] N/A

Deploy method

• Docker[x] Docker Compose[ ] Kubernetes Helm Charts[ ] Kubernetes GMC[ ] Other[ ] N/A

Running nodes

Single Node

What's the version?

afb46bd

Description

https://github.com/opea-project/GenAIExamples/actions/runs/15189596329/job/42752387727

Reproduce steps

cd VideoQnA/tests bash test_compose_on_xeon.sh

Raw log

Attachments

No response

@ZePan110

The log indicates the container micro-service is not started.
echo "Condition not met for $item, keeping in list." >> ${LOG_PATH}/list_check.log

Would you please help to add the log of lvm-video-llama container?

GenAIExamples/VideoQnA/tests/test_compose_on_xeon.sh

Line 129 in 162f5a8

check_condition() {

151         # Iterate through the list
152         for i in "${!list[@]}"; do
153             item=${list[i]}
154             if check_condition "$item"; then
155                 echo "Condition met for $item, removing from list." >> ${LOG_PATH}/list_check.log
156                 unset list[i]
157             else
158                 echo "Condition not met for $item, keeping in list." >> ${LOG_PATH}/list_check.log
159             fi
160         done

[ZePan110]

@xiguiw docker logs lvm-video-llama llm_download: True Please wait for model download... Git LFS initialized. Cloning into '/home/user/model/Video-LLaMA-2-7B-Finetuned'... Filtering content: 100% (6/6), 5.52 GiB | 67.17 MiB/s, done. /usr/local/lib/python3.11/site-packages/timm/models/hub.py:4: FutureWarning: Importing from timm.models.hub is deprecated, please import via timm.models warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.models", FutureWarning) /usr/local/lib/python3.11/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning) /usr/local/lib/python3.11/site-packages/timm/models/registry.py:4: FutureWarning: Importing from timm.models.registry is deprecated, please import via timm.models warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.models", FutureWarning) /usr/local/lib/python3.11/site-packages/torchvision/transforms/_functional_video.py:6: UserWarning: The 'torchvision.transforms._functional_video' module is deprecated since 0.12 and will be removed in the future. Please use the 'torchvision.transforms.functional' module instead. warnings.warn( /usr/local/lib/python3.11/site-packages/torchvision/transforms/_transforms_video.py:22: UserWarning: The 'torchvision.transforms._transforms_video' module is deprecated since 0.12 and will be removed in the future. Please use the 'torchvision.transforms' module instead. warnings.warn( /home/user/comps/third_parties/video-llama/src/server.py:108: LangChainDeprecationWarning: The class HuggingFaceEmbeddings was deprecated in LangChain 0.2.2 and will be removed in 1.0. An updated version of the class exists in the :class:~langchain-huggingface package and should be used instead. To use it run pip install -U :class:~langchain-huggingfaceand import asfrom :class:~langchain_huggingface import HuggingFaceEmbeddings``. HFembeddings = HuggingFaceEmbeddings(model_kwargs={"device": "cpu"}) /home/user/comps/third_parties/video-llama/src/server.py:108: LangChainDeprecationWarning: Default values for HuggingFaceEmbeddings.model_name were deprecated in LangChain 0.2.16 and will be removed in 0.4.0. Explicitly pass a model_name to the HuggingFaceEmbeddings constructor instead. HFembeddings = HuggingFaceEmbeddings(model_kwargs={"device": "cpu"}) INFO:sentence_transformers.SentenceTransformer:Load pretrained SentenceTransformer: sentence-transformers/all-mpnet-base-v2 INFO:faiss.loader:Loading faiss with AVX512 support. INFO:faiss.loader:Successfully loaded faiss with AVX512 support. INFO:faiss:Failed to load GPU Faiss: name 'GpuIndexIVFFlat' is not defined. Will not load constructor refs for GPU indexes. This is only an error if you're trying to use GPU Faiss. /home/user/comps/third_parties/video-llama/src/video_llama/models/eva_vit.py:433: FutureWarning: You are using torch.loadwithweights_only=False(the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value forweights_onlywill be flipped toTrue. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via torch.serialization.add_safe_globals. We recommend you start setting weights_only=Truefor any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature. state_dict = torch.load(cached_file, map_location="cpu") INFO:root:freeze vision encoder BertLMHeadModel has generative capabilities, asprepare_inputs_for_generationis explicitly defined. However, it doesn't directly inherit fromGenerationMixin. From 👉v4.50👈 onwards, PreTrainedModelwill NOT inherit fromGenerationMixin, and this model will lose the ability to call generate` and other related functions.

• If you're using trust_remote_code=True, you can get rid of this warning by loading the model with an auto class. See https://huggingface.co/docs/transformers/en/model_doc/auto#auto-classes
• If you are the owner of the model architecture code, please modify your model class such that it inherits from GenerationMixin (after PreTrainedModel, otherwise you'll get an exception).
• If you are not the owner of the model architecture class, please contact the model code owner to update it.
  /home/user/comps/third_parties/video-llama/src/video_llama/models/blip2.py:78: FutureWarning: You are using torch.load with weights_only=False (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for weights_only will be flipped to True. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via torch.serialization.add_safe_globals. We recommend you start setting weights_only=True for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
  checkpoint = torch.load(cached_file, map_location="cpu")
  INFO:root:load checkpoint from https://storage.googleapis.com/sfr-vision-language-research/LAVIS/models/BLIP2/blip2_pretrained_flant5xxl.pth
  INFO:root:freeze Qformer
  You are using the default legacy behaviour of the <class 'transformers.models.llama.tokenization_llama.LlamaTokenizer'>. This is expected, and simply means that the legacy (previous) behavior will be used so nothing changes for you. If you want to use the new behaviour, set legacy=False. This should only be set if you understand what it means, and thoroughly read the reason why this was added as explained in ⚠️⚠️[T5Tokenize] Fix T5 family tokenizers⚠️⚠️ huggingface/transformers#24565 - if you loaded a llama tokenizer from a GGUF file you can ignore this message
  LlamaForCausalLM has generative capabilities, as prepare_inputs_for_generation is explicitly defined. However, it doesn't directly inherit from GenerationMixin. From 👉v4.50👈 onwards, PreTrainedModel will NOT inherit from GenerationMixin, and this model will lose the ability to call generate and other related functions.
• If you're using trust_remote_code=True, you can get rid of this warning by loading the model with an auto class. See https://huggingface.co/docs/transformers/en/model_doc/auto#auto-classes
• If you are the owner of the model architecture code, please modify your model class such that it inherits from GenerationMixin (after PreTrainedModel, otherwise you'll get an exception).
• If you are not the owner of the model architecture class, please contact the model code owner to update it.
  Seed: 10
  vis_processor vit_precision: fp16
  WARNING! FP16 not currently supported. Switching to FP32
  Loading VIT
  Loading VIT Done
  Loading Q-Former
  Loading Q-Former Done
  Loading LLAMA Tokenizer
  Loading LLAMA Model
  Loading checkpoint shards: 0%| | 0/2 [00:00<?, ?it/s]
  Traceback (most recent call last):
  File "/home/user/comps/third_parties/video-llama/src/server.py", line 281, in
  video_llama = VL(cfg_path=CFG_PATH, model_type=MODEL_TYPE)
  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/comps/third_parties/video-llama/src/extract_vl_embedding.py", line 29, in init
  self.model = model_cls.from_config(model_config).to("cpu")
  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/comps/third_parties/video-llama/src/video_llama/models/video_llama.py", line 588, in from_config
  model = cls(
  ^^^^
  File "/home/user/comps/third_parties/video-llama/src/video_llama/models/video_llama.py", line 142, in init
  self.llama_model = LlamaForCausalLM.from_pretrained(
  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/transformers/modeling_utils.py", line 309, in _wrapper
  return func(*args, **kwargs)
  ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/transformers/modeling_utils.py", line 4573, in from_pretrained
  ) = cls._load_pretrained_model(
  ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/transformers/modeling_utils.py", line 5019, in _load_pretrained_model
  state_dict = load_state_dict(
  ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/transformers/modeling_utils.py", line 553, in load_state_dict
  check_torch_load_is_safe()
  File "/usr/local/lib/python3.11/site-packages/transformers/utils/import_utils.py", line 1417, in check_torch_load_is_safe
  raise ValueError(
  ValueError: Due to a serious vulnerability issue in torch.load, even with weights_only=True, we now require users to upgrade torch to at least v2.6 in order to use the function. This version restriction does not apply when loading files with safetensors.
  See the vulnerability report here https://nvd.nist.gov/vuln/detail/CVE-2025-32434
  llm_download: True
  Please wait for model download...
  Git LFS initialized.
  Cloning into '/home/user/model/Video-LLaMA-2-7B-Finetuned'...
  `
  There seems to be a lack of space

[xiguiw]

• ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/comps/third_parties/video-llama/src/extract_vl_embedding.py", line 29, in init
  self.model = model_cls.from_config(model_config).to("cpu")
  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/comps/third_parties/video-llama/src/video_llama/models/video_llama.py", line 588, in from_config
  model = cls(
  ^^^^
  File "/home/user/comps/third_parties/video-llama/src/video_llama/models/video_llama.py", line 142, in init
  self.llama_model = LlamaForCausalLM.from_pretrained(
  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/transformers/modeling_utils.py", line 309, in _wrapper
  return func(*args, **kwargs)
  ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/transformers/modeling_utils.py", line 4573, in from_pretrained
  ) = cls._load_pretrained_model(
  ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/transformers/modeling_utils.py", line 5019, in _load_pretrained_model
  state_dict = load_state_dict(
  ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/transformers/modeling_utils.py", line 553, in load_state_dict
  check_torch_load_is_safe()
  File "/usr/local/lib/python3.11/site-packages/transformers/utils/import_utils.py", line 1417, in check_torch_load_is_safe
  raise ValueError(
  ValueError: Due to a serious vulnerability issue in torch.load, even with weights_only=True, we now require users to upgrade torch to at least v2.6 in order to use the function. This version restriction does not apply when loading files with safetensors.
  See the vulnerability report here https://nvd.nist.gov/vuln/detail/CVE-2025-32434
  llm_download: True
  Please wait for model download...
  Git LFS initialized.
  Cloning into '/home/user/model/Video-LLaMA-2-7B-Finetuned'...
  `
  There seems to be a lack of space

@ZePan110

Yes.

1. transformers Load model failed.
2. There is not space on disk.

Could this be reproduced after cleaning the disk?

[ZePan110]

llm_download: True Please wait for model download... Git LFS initialized. Cloning into '/home/user/model/Video-LLaMA-2-7B-Finetuned'... Filtering content: 50% (3/6), 253.73 MiB | 82.15 MiB/s Filtering content: 66% (4/6), 253.73 MiB | 82.15 MiB/s Filtering content: 66% (4/6), 3.76 GiB | 154.62 MiB/s Filtering content: 83% (5/6), 3.76 GiB | 154.62 MiB/s Filtering content: 83% (5/6), 4.23 GiB | 7.28 MiB/s Filtering content: 100% (6/6), 4.23 GiB | 7.28 MiB/s Filtering content: 100% (6/6), 5.52 GiB | 24.57 MiB/s Filtering content: 100% (6/6), 5.52 GiB | 88.63 MiB/s, done. /usr/local/lib/python3.11/site-packages/timm/models/hub.py:4: FutureWarning: Importing from timm.models.hub is deprecated, please import via timm.models warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.models", FutureWarning) /usr/local/lib/python3.11/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning) /usr/local/lib/python3.11/site-packages/timm/models/registry.py:4: FutureWarning: Importing from timm.models.registry is deprecated, please import via timm.models warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.models", FutureWarning) /usr/local/lib/python3.11/site-packages/torchvision/transforms/_functional_video.py:6: UserWarning: The 'torchvision.transforms._functional_video' module is deprecated since 0.12 and will be removed in the future. Please use the 'torchvision.transforms.functional' module instead. warnings.warn( /usr/local/lib/python3.11/site-packages/torchvision/transforms/_transforms_video.py:22: UserWarning: The 'torchvision.transforms._transforms_video' module is deprecated since 0.12 and will be removed in the future. Please use the 'torchvision.transforms' module instead. warnings.warn( /home/user/comps/third_parties/video-llama/src/server.py:108: LangChainDeprecationWarning: The class HuggingFaceEmbeddings was deprecated in LangChain 0.2.2 and will be removed in 1.0. An updated version of the class exists in the :class:~langchain-huggingface package and should be used instead. To use it run pip install -U :class:~langchain-huggingfaceand import asfrom :class:~langchain_huggingface import HuggingFaceEmbeddings``. HFembeddings = HuggingFaceEmbeddings(model_kwargs={"device": "cpu"}) /home/user/comps/third_parties/video-llama/src/server.py:108: LangChainDeprecationWarning: Default values for HuggingFaceEmbeddings.model_name were deprecated in LangChain 0.2.16 and will be removed in 0.4.0. Explicitly pass a model_name to the HuggingFaceEmbeddings constructor instead. HFembeddings = HuggingFaceEmbeddings(model_kwargs={"device": "cpu"}) INFO:sentence_transformers.SentenceTransformer:Load pretrained SentenceTransformer: sentence-transformers/all-mpnet-base-v2 INFO:faiss.loader:Loading faiss with AVX512 support. INFO:faiss.loader:Successfully loaded faiss with AVX512 support. INFO:faiss:Failed to load GPU Faiss: name 'GpuIndexIVFFlat' is not defined. Will not load constructor refs for GPU indexes. This is only an error if you're trying to use GPU Faiss. /home/user/comps/third_parties/video-llama/src/video_llama/models/eva_vit.py:433: FutureWarning: You are using torch.loadwithweights_only=False(the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value forweights_onlywill be flipped toTrue. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via torch.serialization.add_safe_globals. We recommend you start setting weights_only=Truefor any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature. state_dict = torch.load(cached_file, map_location="cpu") INFO:root:freeze vision encoder BertLMHeadModel has generative capabilities, asprepare_inputs_for_generationis explicitly defined. However, it doesn't directly inherit fromGenerationMixin. From 👉v4.50👈 onwards, PreTrainedModelwill NOT inherit fromGenerationMixin, and this model will lose the ability to call generate` and other related functions.

• If you're using trust_remote_code=True, you can get rid of this warning by loading the model with an auto class. See https://huggingface.co/docs/transformers/en/model_doc/auto#auto-classes
• If you are the owner of the model architecture code, please modify your model class such that it inherits from GenerationMixin (after PreTrainedModel, otherwise you'll get an exception).
• If you are not the owner of the model architecture class, please contact the model code owner to update it.
  /home/user/comps/third_parties/video-llama/src/video_llama/models/blip2.py:78: FutureWarning: You are using torch.load with weights_only=False (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for weights_only will be flipped to True. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via torch.serialization.add_safe_globals. We recommend you start setting weights_only=True for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
  checkpoint = torch.load(cached_file, map_location="cpu")
  INFO:root:load checkpoint from https://storage.googleapis.com/sfr-vision-language-research/LAVIS/models/BLIP2/blip2_pretrained_flant5xxl.pth
  INFO:root:freeze Qformer
  You are using the default legacy behaviour of the <class 'transformers.models.llama.tokenization_llama.LlamaTokenizer'>. This is expected, and simply means that the legacy (previous) behavior will be used so nothing changes for you. If you want to use the new behaviour, set legacy=False. This should only be set if you understand what it means, and thoroughly read the reason why this was added as explained in ⚠️⚠️[T5Tokenize] Fix T5 family tokenizers⚠️⚠️ huggingface/transformers#24565 - if you loaded a llama tokenizer from a GGUF file you can ignore this message
  LlamaForCausalLM has generative capabilities, as prepare_inputs_for_generation is explicitly defined. However, it doesn't directly inherit from GenerationMixin. From 👉v4.50👈 onwards, PreTrainedModel will NOT inherit from GenerationMixin, and this model will lose the ability to call generate and other related functions.
• If you're using trust_remote_code=True, you can get rid of this warning by loading the model with an auto class. See https://huggingface.co/docs/transformers/en/model_doc/auto#auto-classes
• If you are the owner of the model architecture code, please modify your model class such that it inherits from GenerationMixin (after PreTrainedModel, otherwise you'll get an exception).
• If you are not the owner of the model architecture class, please contact the model code owner to update it.
  Seed: 10
  vis_processor vit_precision: fp16
  WARNING! FP16 not currently supported. Switching to FP32
  Loading VIT
  Loading VIT Done
  Loading Q-Former
  Loading Q-Former Done
  Loading LLAMA Tokenizer
  Loading LLAMA Model

Loading checkpoint shards: 0%| | 0/2 [00:00<?, ?it/s]
Loading checkpoint shards: 0%| | 0/2 [00:00<?, ?it/s]
Traceback (most recent call last):
File "/home/user/comps/third_parties/video-llama/src/server.py", line 281, in
video_llama = VL(cfg_path=CFG_PATH, model_type=MODEL_TYPE)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/comps/third_parties/video-llama/src/extract_vl_embedding.py", line 29, in init
self.model = model_cls.from_config(model_config).to("cpu")
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/comps/third_parties/video-llama/src/video_llama/models/video_llama.py", line 588, in from_config
model = cls(
^^^^
File "/home/user/comps/third_parties/video-llama/src/video_llama/models/video_llama.py", line 142, in init
self.llama_model = LlamaForCausalLM.from_pretrained(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/transformers/modeling_utils.py", line 309, in _wrapper
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/transformers/modeling_utils.py", line 4573, in from_pretrained
) = cls._load_pretrained_model(
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/transformers/modeling_utils.py", line 5019, in _load_pretrained_model
state_dict = load_state_dict(
^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/transformers/modeling_utils.py", line 553, in load_state_dict
check_torch_load_is_safe()
File "/usr/local/lib/python3.11/site-packages/transformers/utils/import_utils.py", line 1417, in check_torch_load_is_safe
raise ValueError(
ValueError: Due to a serious vulnerability issue in torch.load, even with weights_only=True, we now require users to upgrade torch to at least v2.6 in order to use the function. This version restriction does not apply when loading files with safetensors.
See the vulnerability report here https://nvd.nist.gov/vuln/detail/CVE-2025-32434
`
This issue is not related to disk space, I have captured the complete log, which has been repeatedly displaying the above information

[xiguiw]

File "/home/user/comps/third_parties/video-llama/src/video_llama/models/video_llama.py", line 142, in init
self.llama_model = LlamaForCausalLM.from_pretrained(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/transformers/modeling_utils.py", line 309, in _wrapper
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/transformers/modeling_utils.py", line 4573, in from_pretrained
) = cls._load_pretrained_model(
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/transformers/modeling_utils.py", line 5019, in _load_pretrained_model
state_dict = load_state_dict(
^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/transformers/modeling_utils.py", line 553, in load_state_dict
check_torch_load_is_safe()
File "/usr/local/lib/python3.11/site-packages/transformers/utils/import_utils.py", line 1417, in check_torch_load_is_safe
raise ValueError(
ValueError: Due to a serious vulnerability issue in torch.load, even with weights_only=True, we now require users to upgrade torch to at least v2.6 in order to use the function. This version restriction does not apply when loading files with safetensors.
See the vulnerability report here https://nvd.nist.gov/vuln/detail/CVE-2025-32434

The error you're seeing is caused by a security restriction introduced in recent versions of the Hugging Face transformers library due to a serious vulnerability in torch.load. Here's a breakdown:

PyTorch torch.load() (even with weights_only=True) has a known vulnerability:
CVE-2025-32434
This allows execution of arbitrary code during deserialization if the model file is malicious.

Need to upgrade PyTorch to v2.6.0 or higher

[xiguiw]

Need to upgrade PyTorch to v2.6.0 or higher

1. Afte upgrade PyTorch to 2.7, another issue of transformers version changes trigger.

raceback (most recent call last):
  File "/usr/local/lib/python3.11/threading.py", line 1045, in _bootstrap_inner
    self.run()
  File "/usr/local/lib/python3.11/threading.py", line 982, in run
    self._target(*self._args, **self._kwargs)
  File "/home/user/comps/third_parties/video-llama/src/server.py", line 137, in inference
    chat.answer(
  File "/home/user/comps/third_parties/video-llama/src/video_llama/conversation/conversation_video.py", line 224, in answer
    outputs = self.model.llama_model.generate(
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1940, in __getattr__
    raise AttributeError(
AttributeError: 'LlamaForCausalLM' object has no attribute 'generate'
``

From Hugging Face transformers v4.50，PreTrainedModel does not derivate the `GenerationMixin` as a default.

2. the third party fixed at specific commit
 https://github.com/opea-project/GenAIComps/blob/4264cca4e4e370929ed4672972899442bcb94cf9/comps/third_parties/video-llama/src/Dockerfile#L33

So either upgrade the third party video-llama repo commit/version to change the way to load model, or keep transformers versions befor v4.50.