Mount failed for \mnt\opea-models

[ezelanza]

Hi, I've followed the instructions to create a Kubernetes cluster on Xeon without GMC : https://github.com/opea-project/GenAIExamples/blob/main/ChatQnA/kubernetes/manifests/README.md

My environment :
-AWS EKS 1.30
-Node : M7i instances

kubectl get pods
NAME                                       READY   STATUS              RESTARTS         AGE
chatqna-79d8c5ffff-gl44s                   1/1     Running             0                94m
chatqna-data-prep-59cbff99d7-chct8         1/1     Running             0                94m
chatqna-embedding-usvc-97f8f5f5f-zrxlf     1/1     Running             0                94m
chatqna-llm-uservice-7f744cd68b-4kwzt      0/1     Running             8 (10m ago)      94m
chatqna-redis-vector-db-5dcd98f579-bpjtc   1/1     Running             0                94m
chatqna-reranking-usvc-8666ffcf65-tmn9s    1/1     Running             0                94m
chatqna-retriever-usvc-77b545d47f-n2cqp    0/1     Running             17 (7m21s ago)   94m
chatqna-tei-74b9fd8f64-mrhjd               0/1     ContainerCreating   0                94m
chatqna-teirerank-68b7794ff6-mfw67         0/1     ContainerCreating   0                94m
chatqna-tgi-bf455497b-jfxkr                0/1     ContainerCreating   0                94m

Logs

kubectl describe pod chatqna-tgi-bf455497b-jfxkr
Name:             chatqna-tgi-bf455497b-jfxkr
Namespace:        default
Priority:         0
Service Account:  default
Node:             ip-172-31-66-47.ec2.internal/172.31.66.47
Start Time:       Fri, 02 Aug 2024 13:31:48 -0400
Labels:           app.kubernetes.io/instance=chatqna
                  app.kubernetes.io/name=tgi
                  pod-template-hash=bf455497b
Annotations:      <none>
Status:           Pending
IP:               
IPs:              <none>
Controlled By:    ReplicaSet/chatqna-tgi-bf455497b
Containers:
  tgi:
    Container ID:   
    Image:          ghcr.io/huggingface/text-generation-inference:2.1.0
    Image ID:       
    Port:           2080/TCP
    Host Port:      0/TCP
    State:          Waiting
      Reason:       ContainerCreating
    Ready:          False
    Restart Count:  0
    Environment Variables from:
      chatqna-tgi-config  ConfigMap  Optional: false
    Environment:          <none>
    Mounts:
      /data from model-volume (rw)
      /tmp from tmp (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-mmwgw (ro)
Conditions:
  Type                        Status
  PodReadyToStartContainers   False 
  Initialized                 True 
  Ready                       False 
  ContainersReady             False 
  PodScheduled                True 
Volumes:
  model-volume:
    Type:          HostPath (bare host directory volume)
    Path:          /mnt/opea-models
    HostPathType:  Directory
  tmp:
    Type:       EmptyDir (a temporary directory that shares a pod's lifetime)
    Medium:     
    SizeLimit:  <unset>
  kube-api-access-mmwgw:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   BestEffort
Node-Selectors:              <none>
Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type     Reason       Age                   From     Message
  ----     ------       ----                  ----     -------
  Warning  FailedMount  2m30s (x53 over 94m)  kubelet  MountVolume.SetUp failed for volume "model-volume" : hostPath type check failed: /mnt/opea-models is not a directory

FYI

curl http://localhost:8888/v1/chatqna \
    -H 'Content-Type: application/json' \
    -d '{"messages": "What is the revenue of Nike in 2023?"}'
Internal Server Error

[lianhao]

Please pay attention to the following NOTES from README:

You need to make sure you have created the directory /mnt/opea-models to save the cached model on the node where the ChatQnA workload is running. Otherwise, you need to modify the chatqna.yaml file to change the model-volume to a directory that exists on the node.

[ezelanza]

Please pay attention to the following NOTES from README:

You need to make sure you have created the directory /mnt/opea-models to save the cached model on the node where the ChatQnA workload is running. Otherwise, you need to modify the chatqna.yaml file to change the model-volume to a directory that exists on the node.

Thanks, it worked! :)