Adding files to deploy CodeTrans application on ROCm vLLM#1545

Merged

chensuyue merged 33 commits into

opea-project:mainfrom

chyundunovDatamonsters:feature/Codetrans_vLLM

Mar 24, 2025

Contributor

chyundunovDatamonsters commented Feb 13, 2025

Description

Adding files to deploy CodeTrans application on ROCm vLLM.

Issues

It is required to provide the ability to deploy the CodeTrans application on ROCm vLLM

Type of change

List the type of change like below. Please delete options that are not relevant.

[*] New feature (non-breaking change which adds new functionality)

Dependencies

Tests

Testing was performed manually and by running a script CodeTrans/tests/test_compose_vllm_on_rocm.sh This script is similar to the script that runs CodeTrans testing on an Intel CPU.

chyundunovDatamonsters requested review from Spycsh and XinyaoWa as code owners

February 13, 2025 08:29

github-actions Bot commented Feb 13, 2025 •

edited

Loading

Dependency Review

✅ No vulnerabilities or license issues found.

Scanned Files

AlexHe99 suggested changes

View reviewed changes

AlexHe99 left a comment

please refer to my comments in review.

CodeTrans/Dockerfile-vllm-rocm Outdated

CodeTrans/Dockerfile-vllm-rocm Outdated

CodeTrans/Dockerfile-vllm-rocm Outdated

CodeTrans/Dockerfile-vllm-rocm Outdated

CodeTrans/docker_compose/amd/gpu/rocm/compose_vllm.yaml

CodeTrans/docker_compose/amd/gpu/rocm/set_env_vllm.sh Outdated

CodeTrans/tests/test_compose_vllm_on_rocm.sh

louie-tsai and others added 10 commits

February 17, 2025 11:06


          Add a new section to change LLM model such as deepseek based on valid…

097dcaf

…ated model table in LLM microservice (opea-project#1501)

Signed-off-by: Tsai, Louie <louie.tsai@intel.com>
Co-authored-by: Wang, Kai Lawrence <109344418+wangkl2@users.noreply.github.com>
Co-authored-by: xiguiw <111278656+xiguiw@users.noreply.github.com>
Signed-off-by: Chingis Yundunov <YundunovCN@sibedge.com>


          CodeTrans - add deploy app with vLLM ROCm

aa5a93e

Signed-off-by: Chingis Yundunov <YundunovCN@sibedge.com>


          CodeTrans - add deploy app with vLLM ROCm

488aee3

Signed-off-by: Chingis Yundunov <YundunovCN@sibedge.com>


          CodeTrans - add deploy app with vLLM ROCm

3243f4b

Signed-off-by: Chingis Yundunov <YundunovCN@sibedge.com>


          CodeTrans - add deploy app with vLLM ROCm

f5430e5

Signed-off-by: Chingis Yundunov <YundunovCN@sibedge.com>


          CodeTrans - fix Dockerfile for vLLM

1106d0e

Signed-off-by: Chingis Yundunov <YundunovCN@sibedge.com>


          CodeTrans - fix Dockerfile for vLLM

265d67b

Signed-off-by: Chingis Yundunov <YundunovCN@sibedge.com>


          CodeTrans - fix files for deploy with ROCm vLLM

bea90fa

Signed-off-by: Chingis Yundunov <YundunovCN@sibedge.com>


          CodeTrans - fix files for deploy with ROCm vLLM

Signed-off-by: Chingis Yundunov <YundunovCN@sibedge.com>


          CodeTrans - fix files for deploy with ROCm vLLM

470d88b

Signed-off-by: Chingis Yundunov <YundunovCN@sibedge.com>

chyundunovDatamonsters force-pushed the feature/Codetrans_vLLM branch from 66b3d46 to 470d88b Compare

February 17, 2025 04:06

chyundunovDatamonsters requested review from letonghan and lvliang-intel as code owners

February 17, 2025 04:06

Chingis Yundunov added 2 commits

February 17, 2025 11:13


          CodeTrans - fix files for deploy with ROCm vLLM

a3fe495

Signed-off-by: Chingis Yundunov <YundunovCN@sibedge.com>


          CodeTrans - fix files for deploy with ROCm vLLM

cc3ab59

Signed-off-by: Chingis Yundunov <YundunovCN@sibedge.com>

joshuayao mentioned this pull request

[Feature] vLLM enablement for 8 GenAI examples #1436

Closed

21 tasks

chensuyue reviewed

View reviewed changes

CodeTrans/Dockerfile-vllm-rocm Outdated


          CodeTrans - fix ROCm docker compose file

26a443c

Signed-off-by: Chingis Yundunov <YundunovCN@sibedge.com>

Contributor Author

chyundunovDatamonsters commented Mar 6, 2025

@chensuyue, can you please tell me why this check fails - https://github.com/opea-project/GenAIExamples/actions/runs/13698991369/job/38307896179?pr=1545?

joshuayao added this to the v1.3 milestone

Chingis Yundunov and others added 6 commits

March 17, 2025 12:23


          CodeTrans - fix files for deploy on ROCm

7a317ec

Signed-off-by: Chingis Yundunov <YundunovCN@sibedge.com>


          CodeTrans - fix files for deploy on ROCm

19bc3c5

Signed-off-by: Chingis Yundunov <YundunovCN@sibedge.com>


          Merge branch 'main' into feature/Codetrans_vLLM

9feb285


          [pre-commit.ci] auto fixes from pre-commit.com hooks

2d0602c

for more information, see https://pre-commit.ci


          CodeTrans - fix files for deploy on ROCm

061db6c

Signed-off-by: Chingis Yundunov <YundunovCN@sibedge.com>


          [pre-commit.ci] auto fixes from pre-commit.com hooks

3b0e35e

for more information, see https://pre-commit.ci

Chingis Yundunov and others added 9 commits

March 18, 2025 15:19


          CodeTrans - fix files for deploy on ROCm

0d92a20

Signed-off-by: Chingis Yundunov <YundunovCN@sibedge.com>


          Merge remote-tracking branch 'origin/feature/Codetrans_vLLM' into fea…

acf55d9

…ture/Codetrans_vLLM


          CodeTrans - fix files for deploy on ROCm

3db4ac4

Signed-off-by: Chingis Yundunov <YundunovCN@sibedge.com>


          CodeTrans - fix files for deploy on ROCm

204e3a7

Signed-off-by: Chingis Yundunov <YundunovCN@sibedge.com>


          CodeTrans - fix files for deploy on ROCm

f597a85

Signed-off-by: Chingis Yundunov <YundunovCN@sibedge.com>


          [pre-commit.ci] auto fixes from pre-commit.com hooks

4f577ad

for more information, see https://pre-commit.ci


          CodeTrans - fix files for deploy on ROCm

92a8034

Signed-off-by: Chingis Yundunov <YundunovCN@sibedge.com>


          Merge remote-tracking branch 'origin/feature/Codetrans_vLLM' into fea…

524421a

…ture/Codetrans_vLLM


          [pre-commit.ci] auto fixes from pre-commit.com hooks

289cffd

for more information, see https://pre-commit.ci

Collaborator

letonghan commented Mar 18, 2025

Hi @chyundunovDatamonsters , I'm working on merging CodeTrans and Translation example in this PR. I wonder if you can apply your changes in Translation example, so that we can merge your PR with mine automatically?

Chingis Yundunov and others added 5 commits

March 19, 2025 12:24


          CodeTrans - fix files for deploy on ROCm

7415b59

Signed-off-by: Chingis Yundunov <YundunovCN@sibedge.com>


          CodeTrans - fix files for deploy on ROCm

cf68881

Signed-off-by: Chingis Yundunov <YundunovCN@sibedge.com>


          CodeTrans - fix files for deploy on ROCm

e011a9b

Signed-off-by: Chingis Yundunov <YundunovCN@sibedge.com>


          [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci


          Merge branch 'main' into feature/Codetrans_vLLM

b21ed34

chensuyue approved these changes

View reviewed changes

XuehaoSun approved these changes

View reviewed changes

letonghan approved these changes

View reviewed changes

Collaborator

letonghan left a comment

LGTM, the example merge of CodeTrans & Translation is pending for now

chensuyue merged commit a04463d into opea-project:main

chyundunovDatamonsters added a commit to chyundunovDatamonsters/OPEA-GenAIExamples that referenced this pull request


          Adding files to deploy CodeTrans application on ROCm vLLM (opea-proje…

adac5e8

…ct#1545)

Signed-off-by: Tsai, Louie <louie.tsai@intel.com>
Signed-off-by: Chingis Yundunov <YundunovCN@sibedge.com>

chyundunovDatamonsters added a commit to chyundunovDatamonsters/OPEA-GenAIExamples that referenced this pull request


          Adding files to deploy CodeTrans application on ROCm vLLM (opea-proje…

6aebfa4

…ct#1545)

Signed-off-by: Tsai, Louie <louie.tsai@intel.com>
Signed-off-by: Chingis Yundunov <YundunovCN@sibedge.com>

chyundunovDatamonsters added a commit to chyundunovDatamonsters/OPEA-GenAIExamples that referenced this pull request


          Adding files to deploy CodeTrans application on ROCm vLLM (opea-proje…

ebcaf1a

…ct#1545)

Signed-off-by: Tsai, Louie <louie.tsai@intel.com>
Signed-off-by: Chingis Yundunov <YundunovCN@sibedge.com>
Signed-off-by: Chingis Yundunov <c.yundunov@datamonsters.com>

letonghan pushed a commit that referenced this pull request


          upgrade setuptools version adapt to the latest wheel version (#1545)

a941fc5

Signed-off-by: chensuyue <suyue.chen@intel.com>
Co-authored-by: Abolfazl Shahbazi <12436063+ashahba@users.noreply.github.com>

cogniware-devops pushed a commit to Cogniware-Inc/GenAIExamples that referenced this pull request


          Adding files to deploy CodeTrans application on ROCm vLLM (opea-proje…

02de5b6

…ct#1545)

Signed-off-by: Tsai, Louie <louie.tsai@intel.com>
Signed-off-by: Chingis Yundunov <YundunovCN@sibedge.com>
Signed-off-by: cogniware-devops <ambarish.desai@cogniware.ai>

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

chensuyue chensuyue approved these changes

letonghan letonghan approved these changes

XuehaoSun XuehaoSun approved these changes

Spycsh Awaiting requested review from Spycsh Spycsh is a code owner

XinyaoWa Awaiting requested review from XinyaoWa

lvliang-intel Awaiting requested review from lvliang-intel

+2 more reviewers

AlexHe99 AlexHe99 requested changes

Yu-amd Yu-amd left review comments

Labels

None yet