Tools not working when providing a schema via `format=...`

[AlexysGromard]

Description

When using a function tool with a response model and providing its schema via the format argument, the tool is never invoked. The same code works when format is omitted, but breaks when a schema is passed using model_json_schema() from a Pydantic model.

Code to Reproduce

from ollama import ChatResponse, chat
from pydantic import BaseModel, Field
import json

class AddTwoNumbersOutput(BaseModel):
  """
  Output schema for the add_two_numbers function
  """
  a: int = Field(..., description="The first number",)
  b: int = Field(..., description="The second number",)
  result: int = Field(..., description="The result of the addition",)

def add_two_numbers(a: int, b: int) -> int:
  """
  Add two numbers

  Args:
    a (int): The first number
    b (int): The second number

  Returns:
    int: The sum of the two numbers
  """

  # The cast is necessary as returned tool call arguments don't always conform exactly to schema
  # E.g. this would prevent "what is 30 + 12" to produce '3012' instead of 42
  return int(a) + int(b)


messages = [{'role': 'user', 'content': 'What is three plus one?'}]
print('Prompt:', messages[0]['content'])

available_functions = {
  'add_two_numbers': add_two_numbers,
}

response: ChatResponse = chat(
  'llama3.1',
  messages=messages,
  tools=[add_two_numbers],
  format=AddTwoNumbersOutput.model_json_schema(),
)

if response.message.tool_calls:
  tool = response.message.tool_calls[0]
  # Ensure the function is available, and then call it
  if function_to_call := available_functions.get(tool.function.name):
    print('Calling function:', tool.function.name)
    print('Arguments:', tool.function.arguments)
    output = function_to_call(**tool.function.arguments)
    print('Function output:', output)
  else:
    print('Function', tool.function.name, 'not found')

# Only needed to chat with the model using the tool call results
if response.message.tool_calls:
  # Add the function response to messages for the model to use
  messages.append(response.message)
  messages.append({'role': 'tool', 'content': str(output), 'tool_name': tool.function.name})

  # Get final response from model with function outputs
  final_response = chat('llama3.1', messages=messages)
  print('Final response:', json.dumps(final_response.message.content, indent=2, ensure_ascii=False))

else:
  print('No tool calls returned from model')

Observed Behavior

• With format=...: No tool call is made, response.message.tool_calls is empty.
• Without format=...: The function add_two_numbers is correctly triggered and returns the result.

Expected Behavior

Providing a schema via format=AddTwoNumbersOutput.model_json_schema() should not prevent tool calls from being triggered.

My Environment

• OS: macOS 15.6 (M1)
• Python: 3.13.5
• ollama-python: 0.5.1
• Ollama 0.10.1
• Model: llama3.1

[lennartpollvogt]

I would like to keep it that way. I only use the model schema in format when I want to force the LLM to adhere to it, and it shouldn't have the possibility to call a tool. Otherwise the reliability would decrease.

But maybe you can explain what why you would expect the behavior you've described? Which use cases do you have where this would be necessary?

[zeerd]

The example given is not very good.
In fact, in the vast majority of cases, we do not know when LLM will complete all tool call processes and begin outputting the final results.

The real progress would be like below:

res = chat(tools=[...])
loop tool_calls in res
    function-call
    res = chat(tools=[...]) # key point
end
res = chat(formats=[...])

The last chat(tools=[...]) that give res without tool_calls in the loop will give the final answer without format.
User need to chat again with formats=[...] for the formatted answer.

This is a waste of tokens.

[ParthSareen]

We're probably not going to support structured generation around tool calling for now. It's best to use a model better at tool calling. I will be working on erroring better so that the error can be used to inform the model about the incorrect call.