[Q] Seeking Clarification on History Behavior with StreamContentAsync and Function Calling

[RyanStanLin]

Hi team,

I've been working with the streaming function calling feature in ChatSession recently and have encountered some behavior regarding the History log that I found a bit confusing. I was hoping to ask for some clarification here to see if my understanding is correct or if this might be an unexpected behavior.

.NET SDK:

Version: 9.0.100-rc.2.24474.11
Commit: 315e1305db
Workload version: 9.0.100-manifests.f0b34084
MSBuild version: 17.12.0-preview-24473-03+fea15fbd1

Runtime Environment:

OS Name: Mac OS X
OS Version: 26.0
OS Platform: Darwin
RID: osx-arm64

My Use Case

I'm trying to implement a multi-turn conversation that involves function calls.

Use Demo JavascriptCodeExecutor as an example:

I replaced :

while (true)
{
    Console.WriteLine();
    Console.WriteLine("Input: ");
    var input = Console.ReadLine();
    if(input == "restart")
        goto restart;
    var prompt =
        $"\r\nExecute the javascript code for this task:\r\n\r\n{input}\r\n\r\n";
    
    var request = new GenerateContentRequest();
    request.AddText(input ?? "");
    
    //Generate Response
    var result = await chat.GenerateContentAsync(request).ConfigureAwait(false);
    
    //Print Result
    Console.WriteLine();
    Console.Write("Result from Gemini:\r\n");
    Console.WriteLine(result.Text());
}

With:

while (true)
{
    Console.WriteLine();
    Console.WriteLine("Input: ");
    var input = Console.ReadLine();
    if(input == "restart")
        goto restart;
    var prompt =
        $"\r\nExecute the javascript code for this task:\r\n\r\n{input}\r\n\r\n";
    
    var request = new GenerateContentRequest();
    request.AddText(input ?? "");
    
    //Generate Response Using Stream
    await foreach (var response in chat.StreamContentAsync(request, default))
    {
        if (!string.IsNullOrEmpty(response.Text()))
        {
            Console.Write(response.Text());
        }
    }
}

And I found the same issue

My general workflow is:

First Turn: I send a request that requires a tool call (e.g., "calculate the sum from 1 to 69").

My app receives the functionCall from the model, executes it locally, and sends the functionResponse back.

I then use StreamContentAsync to wait for the model to stream back the final text-based answer.

Second Turn: After the first turn is fully complete, I proceed to ask a follow-up question.

Observed Behavior

What I've noticed is that after the first turn concludes, when I'm about to start the second turn, the internal History of the ChatSession seems to only contain the last two parts of the first interaction: the functionResponse and the model's final text answer.

The conversation history in the JSON request is disorganized and illogical, which becomes particularly problematic when I have many callable functions and the model performs multiple function calls in a single response.

As far as I know, Gemini requires the first or last entry in the contents array to have a specific role; otherwise, it will throw an error.

---

Raw Json Request/Response

First Run Request 1: input "calculate the sum from 1 to 69"

{
  "contents": [
    {
      "parts": [
        {
          "text": "calculate the sum from 1 to 69"
        }
      ],
      "role": "user"
    }
  ],
  "tools": [
    {
      "functionDeclarations": [
        {
          "name": "ExecuteJavascriptCodeAsync",
...

First Run Response 1:

[
  {
    "candidates": [
      {
        "content": {
          "parts": [
            {
              "functionCall": {
                "name": "ExecuteJavascriptCodeAsync",
                "args": {
                  "code": "let sum = 0; for (let i = 1; i <= 69; i++) { sum += i; } console.log(sum);"
                }
              }
            }
          ],
          "role": "model"
        },
        "finishReason": "STOP"
      }
    ],
    "usageMetadata": {
      "promptTokenCount": 93,
      "candidatesTokenCount": 39,
      "totalTokenCount": 132,
      "promptTokensDetails": [
        {
          "modality": "TEXT",
          "tokenCount": 93
        }
      ],
      "candidatesTokensDetails": [
        {
          "modality": "TEXT",
          "tokenCount": 39
        }
      ]
    },
    "modelVersion": "gemini-2.0-flash",
    "responseId": "APd9aMzoJsuO7dcP9PjzmAY"
  }
]

First Run Request 2:

{
  "contents": [
    {
      "parts": [
        {
          "text": "calculate the sum from 1 to 69"
        }
      ],
      "role": "user"
    },
    {
      "parts": [
        {
          "functionCall": {
            "name": "ExecuteJavascriptCodeAsync",
            "args": {
              "code": "let sum = 0; for (let i = 1; i <= 69; i++) { sum += i; } console.log(sum);"
            }
          }
        }
      ],
      "role": "model"
    },
    {
      "parts": [
        {
          "functionResponse": {
            "name": "ExecuteJavascriptCodeAsync",
            "response": {
              "name": "ExecuteJavascriptCodeAsync",
              "content": "2415\n"
            }
          }
        }
      ],
      "role": "function"
    }
  ],
  "tools": [
    {
      "functionDeclarations": [
        {
          "name": "ExecuteJavascriptCodeAsync",
...

First Run Response 2:

[
  {
    "candidates": [
      {
        "content": {
          "parts": [
            {
              "text": "The"
            }
          ],
          "role": "model"
        }
      }
    ],
    "usageMetadata": {
      "promptTokenCount": 187,
      "totalTokenCount": 187,
      "promptTokensDetails": [
        {
          "modality": "TEXT",
          "tokenCount": 187
        }
      ]
    },
    "modelVersion": "gemini-2.0-flash",
    "responseId": "Avd9aPaoJdWU7dcPv9bKkQs"
  },
  {
    "candidates": [
      {
        "content": {
          "parts": [
            {
              "text": " sum from 1 to 69 is 2415.\n"
            }
          ],
          "role": "model"
        }
      }
    ],
    "usageMetadata": {
      "promptTokenCount": 187,
      "totalTokenCount": 187,
      "promptTokensDetails": [
        {
          "modality": "TEXT",
          "tokenCount": 187
        }
      ]
    },
    "modelVersion": "gemini-2.0-flash",
    "responseId": "Avd9aPaoJdWU7dcPv9bKkQs"
  },
  {
    "candidates": [
      {
        "content": {
          "parts": [
            {
              "text": ""
            }
          ],
          "role": "model"
        },
        "finishReason": "STOP"
      }
    ],
    "usageMetadata": {
      "promptTokenCount": 147,
      "candidatesTokenCount": 17,
      "totalTokenCount": 164,
      "promptTokensDetails": [
        {
          "modality": "TEXT",
          "tokenCount": 147
        }
      ],
      "candidatesTokensDetails": [
        {
          "modality": "TEXT",
          "tokenCount": 17
        }
      ]
    },
    "modelVersion": "gemini-2.0-flash",
    "responseId": "Avd9aPaoJdWU7dcPv9bKkQs"
  }
]

Second Run Request 1: input "calculate the sum from 1-42"

{
  "contents": [
    {
      "parts": [//Weird History
        {
          "functionResponse": {
            "name": "ExecuteJavascriptCodeAsync",
            "response": {
              "name": "ExecuteJavascriptCodeAsync",
              "content": "2415\n"
            }
          }
        }
      ],
      "role": "function"
    },
    {
      "parts": [
        {
          "text": "The\n sum from 1 to 69 is 2415.\n\n\n"
        }
      ],
      "role": "model"
    },
    {
      "parts": [
        {
          "text": "calculate the sum from 1 to 69"
        }
      ],
      "role": "user"
    },
    {
      "parts": [
        {
          "text": "\nThe\n sum from 1 to 69 is 2415.\n\n\n"
        }
      ],
      "role": "model"
    },
    {
      "parts": [
        {
          "text": "calculate the sum from 1 to 42"
        }
      ],
      "role": "user"
    }
  ],
  "tools": [
    {
      "functionDeclarations": [
        {
          "name": "ExecuteJavascriptCodeAsync",
...

[RyanStanLin]

Possible Reason for the Issue

I think the root of the problem is the history update logic at the end of the ChatSession.StreamContentAsync method:

var lastRequestContent = request.Contents.Last(); // <-- The fatal flaw: only gets the last part of the request
var lastResponseContent = RequestExtensions.FormatGenerateContentInput(sb.ToString(), Roles.Model);
UpdateHistory(lastRequestContent, lastResponseContent);

In a multi-step function call, the request sent to the model to obtain the final answer contains the entire turn's context: [user_prompt, model_function_call, user_function_response]. The request.Contents.Last() call, therefore, incorrectly retrieves only the user_function_response, not the user's original prompt.

[gunpal5]

I can understand your concern. Chat history will not store any Function related requests or responses. It was a design choice to optimize the context window usage. Some tools like fetching web page or reading files returns large amount of data which consumes the context too much. to avoid this I just ignored the function call related chat history.

If you need all the function related history, I can add a behavior into the SDK.

[RyanStanLin]

I get why function calls are skipped to save context, but the problem is they’re not skipped cleanly — the SDK ends up sending broken or partial function call data in the JSON. Right now it can cause issues.

[gunpal5]

I will look into this. function calls should be entirely ignored.

[RyanStanLin]

Thanks! I've tried to fix it like this:

public override async IAsyncEnumerable<GenerateContentResponse> StreamContentAsync(GenerateContentRequest request, [EnumeratorCancellation] CancellationToken cancellationToken = default)
    {
        var historyCountBeforeRequest = this.History.Count;
        var sb = new StringBuilder();
        await foreach (var response in base.StreamContentAsync(request, cancellationToken).ConfigureAwait(false))
        {
            if (cancellationToken.IsCancellationRequested)
                yield break;
            sb.Append(response.Text());
            yield return response;
        }
        var finalModelResponseContent = RequestExtensions.FormatGenerateContentInput(sb.ToString(), Roles.Model);
        var userPromptContent = request.Contents
            .Skip(historyCountBeforeRequest)
            .FirstOrDefault(c => c.Role?.Equals(Roles.User, StringComparison.OrdinalIgnoreCase) == true && 
                                 c.Parts.All(p => p.FunctionResponse == null));
        if (userPromptContent != null)
        {
            UpdateHistory(userPromptContent, finalModelResponseContent);
        }
        else
        {
            this.LastResponseContent = finalModelResponseContent;
        }
    }

This way I make sure only valid user prompts are recorded, and function calls don’t mess up the history state.
If it helps, I can try putting together a PR.

[gunpal5]

PR will be helpful. Please create a PR