Skip to content

Runner.run_async silently drops function calls for certain image inputs #4483

@arman-instalily

Description

@arman-instalily

Description

When using Runner.run_async() with a tool-calling agent, the runner silently drops function calls for certain image inputs. The model generates valid function_call parts (confirmed via direct genai SDK call), but the runner yields events with empty/no parts and tools are never invoked.

This is input-dependent — the same agent with the same tools works correctly on one image but fails silently on another.

Environment

  • google-adk 1.23.0
  • google-genai (latest)
  • Model: gemini-3-flash-preview
  • Python 3.14
  • macOS (Apple Silicon)

Reproduction

Step 1: Confirm the model generates function calls (direct genai SDK)

import asyncio
from google.genai import types, Client

async def main():
    client = Client()  # uses GEMINI_API_KEY env var

    uploaded = await client.aio.files.upload(
        file="path/to/image.png",
        config=types.UploadFileConfig(mime_type="image/png"),
    )

    tool_declarations = types.Tool(
        function_declarations=[
            types.FunctionDeclaration(
                name="record_text_block",
                description="Record a text block extracted from the document.",
                parameters=types.Schema(
                    type="OBJECT",
                    properties={
                        "content": types.Schema(type="STRING"),
                        "block_type": types.Schema(type="STRING"),
                        "confidence": types.Schema(type="NUMBER"),
                        "region_bbox": types.Schema(
                            type="ARRAY",
                            items=types.Schema(type="INTEGER"),
                        ),
                    },
                    required=["content", "block_type", "confidence", "region_bbox"],
                ),
            ),
        ]
    )

    response = await client.aio.models.generate_content(
        model="gemini-3-flash-preview",
        contents=[
            types.Content(
                role="user",
                parts=[
                    types.Part.from_uri(file_uri=uploaded.uri, mime_type=uploaded.mime_type),
                    types.Part.from_text(text="Extract all content. Call record_text_block for every text region."),
                ],
            )
        ],
        config=types.GenerateContentConfig(
            temperature=0.1,
            tools=[tool_declarations],
        ),
    )

    # This shows 15+ function_call parts — model works correctly
    for part in response.candidates[0].content.parts:
        if part.function_call:
            print(f"FC: {part.function_call.name}")

    await client.aio.files.delete(name=uploaded.name)

asyncio.run(main())

Result: 15-18 function_call parts returned. Model is generating the correct tool calls.

Step 2: Same image through ADK Runner — tools never execute

import asyncio
import uuid
from google.genai import types
from google.adk.agents import Agent
from google.adk.runners import Runner
from google.adk.sessions import InMemorySessionService
from google.adk.tools import ToolContext

def record_text_block(
    content: str,
    block_type: str,
    confidence: float,
    region_bbox: list[int],
    tool_context: ToolContext,
) -> dict:
    """Record a text block extracted from the document."""
    blocks = tool_context.state.get("blocks", [])
    blocks.append({"content": content, "block_type": block_type})
    tool_context.state["blocks"] = blocks
    print(f"TOOL CALLED: record_text_block({content[:50]!r})")  # Never printed for affected images
    return {"status": "recorded"}

agent = Agent(
    name="test_agent",
    model="gemini-3-flash-preview",
    instruction="Extract all content from documents using tools.",
    tools=[record_text_block],
    generate_content_config=types.GenerateContentConfig(temperature=0.1),
)

session_service = InMemorySessionService()
runner = Runner(agent=agent, app_name="test", session_service=session_service)

async def main():
    from google.genai import Client

    client = Client()
    uploaded = await client.aio.files.upload(
        file="path/to/image.png",  # same image as Step 1
        config=types.UploadFileConfig(mime_type="image/png"),
    )

    session = await session_service.create_session(
        app_name="test",
        user_id="user",
        state={"blocks": []},
        session_id=f"test_{uuid.uuid4().hex[:8]}",
    )

    user_message = types.Content(
        role="user",
        parts=[
            types.Part.from_uri(file_uri=uploaded.uri, mime_type=uploaded.mime_type),
            types.Part.from_text(text="Extract all content. Call record_text_block for every text region."),
        ],
    )

    event_count = 0
    async for event in runner.run_async(
        user_id="user",
        session_id=session.id,
        new_message=user_message,
    ):
        event_count += 1
        parts = []
        if event.content and event.content.parts:
            for part in event.content.parts:
                if hasattr(part, "function_call") and part.function_call:
                    parts.append(f"FC({part.function_call.name})")
                elif part.text:
                    parts.append(f"TEXT({len(part.text)})")
                else:
                    parts.append("OTHER")
        print(f"Event {event_count}: parts={parts}")

    final = await session_service.get_session(
        app_name="test", user_id="user", session_id=session.id
    )
    blocks = final.state.get("blocks", [])
    print(f"\nTotal events: {event_count}, Blocks recorded: {len(blocks)}")

    await client.aio.files.delete(name=uploaded.name)

asyncio.run(main())

Result for affected images: 1 event with empty parts, 0 blocks recorded. record_text_block is never called despite the model generating function calls (confirmed in Step 1).

Result for other images (same code, different file): 10+ events with FC parts, 15+ blocks recorded. Tools execute correctly.

Expected behavior

If the model generates function_call parts, the runner should dispatch them to the registered tool functions and yield events containing the function call/response cycle.

Actual behavior

For certain image inputs, Runner.run_async() yields events with zero parts and terminates without ever invoking the tool functions. No error is raised — the failure is completely silent.

Workaround

Reinitialize ADK components (Agent, Runner, SessionService) on each retry and fall back to a direct genai SDK call (bypassing ADK) if all retries fail.

Additional context

  • The issue is input-dependent: same agent + tools + model works on image A but silently fails on image B
  • Adding BuiltInPlanner with ThinkingConfig does not change the behavior
  • The InMemorySessionService state mechanism works correctly when tools are actually invoked
  • We tested with FUNSD dataset images (scanned forms). Image 82092117.png works, 82200067_0069.png fails consistently

Metadata

Metadata

Assignees

No one assigned

    Labels

    core[Component] This issue is related to the core interface and implementation

    Type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions