When using request to vllm backend with stream = true, no actual result is returned

[vshapenko]

LocalAI version:
v2.5.1-21-ge9c3bbc

Environment, CPU architecture, OS, and Version:
Ubuntu Linux 22.04

Describe the bug
When i am making request to vllm backend with stream = true, i don't get actual answer. For example:

POST http://127.0.0.1:8080/v1/chat/completions
Content-Type: application/json

{
    "model":"vllm",
    "temperature":0.6,
    "stream": true,
    "messages": 
    [   
        {"role": "system", "content" : "You are helpful assistant"},
        {"role":"user", "content": "Name 7 wonders of Earth"}
    ]
}

HTTP/1.1 200 OK
Date: Fri, 19 Jan 2024 12:16:11 GMT
Content-Type: text/event-stream
Cache-Control: no-cache
Connection: keep-alive, close
Transfer-Encoding: chunked

data: {"created":1705666552,"object":"chat.completion.chunk","id":"18f3aab7-bcc7-4a4a-a458-a8237d6b97cb","model":"vllm","choices":[{"index":0,"delta":{"role":"assistant","content":""}}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}

data: {"created":1705666552,"object":"chat.completion.chunk","id":"18f3aab7-bcc7-4a4a-a458-a8237d6b97cb","model":"vllm","choices":[{"index":0,"finish_reason":"stop","delta":{"content":""}}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}

data: [DONE]

When i repeat same request without streaming , i get actual answer.

To Reproduce
Install localai locally, take mistral-instruct-7B for vllm, run the request above.

Expected behavior
Vllm backend returns answer on stream = true

Logs
4:16PM DBG GRPC(mistralai/Mistral-7B-Instruct-v0.1-127.0.0.1:46433): stdout INFO 01-19 16:16:31 model_runner.py:547] Graph capturing finished in 4 secs.
Processed prompts: 100%|██████████| 1/1 [00:00<00:00, 4.86it/s]33): stderr
Error rpc error: code = Unknown desc = Exception iterating responses: 'Result' object is not an iterator

Additional context
Also for some reason any vllm model i tried returns very small (~50) number of tokens in response. Do i miss something in my model settings?

Below is context of vllm.yaml:

name: vllm
backend: vllm
context_size: 1024
parameters:
    model: "mistralai/Mistral-7B-Instruct-v0.1"
    top_k: 40
    top_p: 0.95
    batch: 24
    tfz: 1.0
  
template:
    chat_message: vvlm-chat

and template i use

{{if eq .RoleName "assistant"}}{{.Content}}{{else}}
[INST]
{{if .SystemPrompt}}{{.SystemPrompt}}{{else if eq .RoleName "system"}}<<SYS>>{{.Content}}<</SYS>>

{{else if .Content}}{{.Content}}{{end}}
[/INST] 
{{end}}