From b2ca7863e2653e8ae1e55e1cd6d5c4571351a06e Mon Sep 17 00:00:00 2001 From: "Yao, Qing" Date: Thu, 29 Aug 2024 16:15:00 +0800 Subject: [PATCH 1/3] [Codegen] Refine readme to prompt users on how to change the model. Signed-off-by: Yao, Qing --- CodeGen/README.md | 5 +++-- CodeGen/kubernetes/manifests/README.md | 2 ++ 2 files changed, 5 insertions(+), 2 deletions(-) diff --git a/CodeGen/README.md b/CodeGen/README.md index d9b399a35c..de5ced5b14 100644 --- a/CodeGen/README.md +++ b/CodeGen/README.md @@ -56,8 +56,9 @@ To set up environment variables for deploying ChatQnA services, follow these ste 3. Set up other environment variables: > Note: By default, the [`docker/set_env.sh`](docker/set_env.sh) file will configure your environment - > variables to use [meta-llama/CodeLlama-7b-hf](https://huggingface.co/meta-llama/CodeLlama-7b-hf). This - > is a gated model that requires submitting an access request through Hugging Face. + > variable `LLM_MODEL_ID` to use [meta-llama/CodeLlama-7b-hf](https://huggingface.co/meta-llama/CodeLlama-7b-hf). This + > is a gated model that requires submitting an access request through Hugging Face. You can replace it with another model, + > such as: [Qwen/CodeQwen1.5-7B-Chat](https://huggingface.co/Qwen/CodeQwen1.5-7B-Chat), [deepseek-ai/deepseek-coder-6.7b-instruct](https://huggingface.co/deepseek-ai/deepseek-coder-6.7b-instruct) ```bash source ./docker/set_env.sh diff --git a/CodeGen/kubernetes/manifests/README.md b/CodeGen/kubernetes/manifests/README.md index 4e0a0e0b69..87d6490f8a 100644 --- a/CodeGen/kubernetes/manifests/README.md +++ b/CodeGen/kubernetes/manifests/README.md @@ -14,7 +14,9 @@ ``` cd GenAIExamples/CodeGen/kubernetes/manifests/xeon export HUGGINGFACEHUB_API_TOKEN="YourOwnToken" +export MODEL_ID="meta-llama/CodeLlama-7b-hf" sed -i "s/insert-your-huggingface-token-here/${HUGGINGFACEHUB_API_TOKEN}/g" codegen.yaml +sed -i "s/meta-llama\/CodeLlama-7b-hf/${MODEL_ID}/g" codegen.yaml kubectl apply -f codegen.yaml ``` From b452b9cdcb7773dbe9ba626522a9fb01d98e5b9f Mon Sep 17 00:00:00 2001 From: "Yao, Qing" Date: Thu, 29 Aug 2024 19:08:57 +0800 Subject: [PATCH 2/3] [Codegen] Add section Required Model. Signed-off-by: Yao, Qing --- CodeGen/README.md | 17 ++++++++++++----- 1 file changed, 12 insertions(+), 5 deletions(-) diff --git a/CodeGen/README.md b/CodeGen/README.md index de5ced5b14..38b734a852 100644 --- a/CodeGen/README.md +++ b/CodeGen/README.md @@ -32,6 +32,18 @@ Currently we support two ways of deploying ChatQnA services with docker compose: 2. Start services using the docker images built from source. See the [Gaudi Guide](./docker/gaudi/README.md) or [Xeon Guide](./docker/xeon/README.md) for more information. +### Required Models + +By default, the LLM model is set to a default value as listed below: + +| Service | Model | +|---------------|----------------------------------------------------------------------------------| +| LLM_MODEL_ID | [meta-llama/CodeLlama-7b-hf](https://huggingface.co/meta-llama/CodeLlama-7b-hf) | + +[meta-llama/CodeLlama-7b-hf](https://huggingface.co/meta-llama/CodeLlama-7b-hf) is a gated model that requires submitting an access request through Hugging Face. You can replace it with another model. +Change the `LLM_MODEL_ID` below for your needs, such as: [Qwen/CodeQwen1.5-7B-Chat](https://huggingface.co/Qwen/CodeQwen1.5-7B-Chat), [deepseek-ai/deepseek-coder-6.7b-instruct](https://huggingface.co/deepseek-ai/deepseek-coder-6.7b-instruct) + + ### Setup Environment Variable To set up environment variables for deploying ChatQnA services, follow these steps: @@ -55,11 +67,6 @@ To set up environment variables for deploying ChatQnA services, follow these ste 3. Set up other environment variables: - > Note: By default, the [`docker/set_env.sh`](docker/set_env.sh) file will configure your environment - > variable `LLM_MODEL_ID` to use [meta-llama/CodeLlama-7b-hf](https://huggingface.co/meta-llama/CodeLlama-7b-hf). This - > is a gated model that requires submitting an access request through Hugging Face. You can replace it with another model, - > such as: [Qwen/CodeQwen1.5-7B-Chat](https://huggingface.co/Qwen/CodeQwen1.5-7B-Chat), [deepseek-ai/deepseek-coder-6.7b-instruct](https://huggingface.co/deepseek-ai/deepseek-coder-6.7b-instruct) - ```bash source ./docker/set_env.sh ``` From d0c66113cb1d84086b5cdc89b2672cd9b48ca482 Mon Sep 17 00:00:00 2001 From: "pre-commit-ci[bot]" <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Thu, 29 Aug 2024 11:09:26 +0000 Subject: [PATCH 3/3] [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --- CodeGen/README.md | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/CodeGen/README.md b/CodeGen/README.md index 38b734a852..9917b6ad14 100644 --- a/CodeGen/README.md +++ b/CodeGen/README.md @@ -36,14 +36,13 @@ Currently we support two ways of deploying ChatQnA services with docker compose: By default, the LLM model is set to a default value as listed below: -| Service | Model | -|---------------|----------------------------------------------------------------------------------| -| LLM_MODEL_ID | [meta-llama/CodeLlama-7b-hf](https://huggingface.co/meta-llama/CodeLlama-7b-hf) | +| Service | Model | +| ------------ | ------------------------------------------------------------------------------- | +| LLM_MODEL_ID | [meta-llama/CodeLlama-7b-hf](https://huggingface.co/meta-llama/CodeLlama-7b-hf) | [meta-llama/CodeLlama-7b-hf](https://huggingface.co/meta-llama/CodeLlama-7b-hf) is a gated model that requires submitting an access request through Hugging Face. You can replace it with another model. Change the `LLM_MODEL_ID` below for your needs, such as: [Qwen/CodeQwen1.5-7B-Chat](https://huggingface.co/Qwen/CodeQwen1.5-7B-Chat), [deepseek-ai/deepseek-coder-6.7b-instruct](https://huggingface.co/deepseek-ai/deepseek-coder-6.7b-instruct) - ### Setup Environment Variable To set up environment variables for deploying ChatQnA services, follow these steps: