change codegen tgi model#646
Merged
Merged
Conversation
Signed-off-by: KfreeZ <kefei.zhang@intel.com>
Signed-off-by: KfreeZ <kefei.zhang@intel.com>
Signed-off-by: KfreeZ <kefei.zhang@intel.com>
daisy-ycguo
reviewed
Aug 22, 2024
Collaborator
daisy-ycguo
left a comment
daisy-ycguo
left a comment
There was a problem hiding this comment.
Why we use different models for xeon and Gaudi?
Contributor
Author
the tests with same model running on gaudi is faster than on xeon, and in GMC's test on xeon failed with the timeout result. So for not blocking the tests purpose, switch to another model, but I have created an issue opea-project/GenAIInfra#338 to identify whether it's GMC specific and find out the root cause |
Collaborator
|
Good job on filing an issue to investigate more
…________________________________
From: Kefei Zhang ***@***.***>
Sent: Wednesday, August 21, 2024 8:39:39 PM
To: opea-project/GenAIExamples ***@***.***>
Cc: Bhandaru, Malini ***@***.***>; Comment ***@***.***>
Subject: Re: [opea-project/GenAIExamples] change codegen tgi model (PR #646)
Why we use different models for xeon and Gaudi?
the model running on guadi is faster than on xeon, and in GMC's test on xeon failed with the timeout result. So for not blocking the tests, switch to another model, but I have created an issue opea-project/GenAIInfra#338<opea-project/GenAIInfra#338> to identify whether it's GMC specific and find out the root cause
—
Reply to this email directly, view it on GitHub<#646 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AAUTXVM526DLSMAKOSRZUJTZSVMPXAVCNFSM6AAAAABM5FDG5KVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMBTGYYTCNRUHE>.
You are receiving this because you commented.Message ID: ***@***.***>
|
daisy-ycguo
approved these changes
Aug 22, 2024
3 tasks
dmsuehir
pushed a commit
to dmsuehir/GenAIExamples
that referenced
this pull request
Sep 11, 2024
* change codegen tgi model Signed-off-by: KfreeZ <kefei.zhang@intel.com>
wangkl2
pushed a commit
to wangkl2/GenAIExamples
that referenced
this pull request
Dec 11, 2024
* add resume finetuning checkpoint ut. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add final tuned model. --------- Co-authored-by: root <root@idc708073.jf.intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
cogniware-devops
pushed a commit
to Cogniware-Inc/GenAIExamples
that referenced
this pull request
Dec 19, 2025
* change codegen tgi model Signed-off-by: KfreeZ <kefei.zhang@intel.com> Signed-off-by: cogniware-devops <ambarish.desai@cogniware.ai>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Same as #625 mentioned, the tgi 2.2.0 in v0.9 is also not compatible with "ise-uiuc/Magicoder-S-DS-6.7B" model, which GMC e2e is using in the codegen tests.
Update the model to "meta-llama/CodeLlama-7b-hf" in gaudi and "HuggingFaceH4/mistral-7b-grok" in xeon for codegen in GMC pipeline config
Issues
n/a.Type of change
List the type of change like below. Please delete options that are not relevant.
Dependencies
n/a.Tests
CI/CD will cover this