Skip to content

Fix spoken audio responses on Gaudi and add tests#63

Merged
dmsuehir merged 1 commit into
mmqna-phase3from
dina/fix_gaudi_audio_response
Mar 17, 2025
Merged

Fix spoken audio responses on Gaudi and add tests#63
dmsuehir merged 1 commit into
mmqna-phase3from
dina/fix_gaudi_audio_response

Conversation

@dmsuehir
Copy link
Copy Markdown
Collaborator

Description

During MMQnA phase 3 testing on Gaudi, I found that audio responses are not working on Gaudi:

$ curl http://${host_ip}:${MEGA_SERVICE_PORT}/v1/multimodalqna \
        -H "Content-Type: application/json" \
        -d '{"messages": [{"role": "user", "content": [{"type": "text", "text": "hello, "}, {"type": "image_url", "image_url": {"url": "https://www.ilankelman.org/stopsigns/australia.jpg"}}]}, {"role": "assistant", "content": "opea project! "}, {"role": "user", "content": "chao, "}], "max_tokens": 10, "modalities": ["text", "audio"]}'
Internal Server Error

I debugged this and found that the TTS endpoint was incorrect due to a missing env var. Also, we don't have any megaservice tests with "modalities": ["text", "audio"], so I added one for both Xeon and Gaudi.

Issues

Fix for a MMQnA phase 3 feature

Type of change

List the type of change like below. Please delete options that are not relevant.

  • Bug fix (non-breaking change which fixes an issue)

Dependencies

N/A

Tests

Added a test for getting an audio response

Signed-off-by: dmsuehir <dina.s.jones@intel.com>
@dmsuehir dmsuehir requested a review from mhbuehler as a code owner March 17, 2025 20:45
Copy link
Copy Markdown
Owner

@mhbuehler mhbuehler left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@dmsuehir dmsuehir merged commit 401570f into mmqna-phase3 Mar 17, 2025
@dmsuehir dmsuehir deleted the dina/fix_gaudi_audio_response branch March 17, 2025 23:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants