fix(session): retry empty stream truncations with attempt cap#26167
Open
edevil wants to merge 1 commit intoanomalyco:devfrom
Open
fix(session): retry empty stream truncations with attempt cap#26167edevil wants to merge 1 commit intoanomalyco:devfrom
edevil wants to merge 1 commit intoanomalyco:devfrom
Conversation
Detect AI SDK `finishReason="other"` with zero output as upstream stream truncation. Surface as retryable `APIError` tagged `EmptyOther`. Cap retries at 3 attempts so misbehaving providers can't loop forever. Add a `fromError` case for `APIError` class instances so the structured message and metadata are preserved on the assistant message instead of being wrapped in a generic `UnknownError` whose payload is the JSON-stringified original.
Contributor
|
Thanks for your contribution! This PR doesn't have a linked issue. All PRs must reference an existing issue. Please:
See CONTRIBUTING.md for details. |
Contributor
|
The following comment was made by an LLM, it may be inaccurate: ResultsFound 1 related PR:
Note: PR #26167 is the current PR being analyzed, so it correctly appears in search results but is not a duplicate of itself. No other duplicate PRs found addressing the same issue. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Issue for this PR
Closes #26170
Related #21727
Type of change
What does this PR do?
When an upstream provider stream ends without a proper
stop_reason, the AISDK emits a fallback
finishReason: "other"with zero output tokens.opencode previously accepted this as a normal end-of-step, persisting a
truncated message with no error and no retry. The user got a half-finished
response and had to manually re-prompt.
This PR detects the truncation pattern at the session-processor layer and
surfaces it as a retryable
APIError, capped at 3 attempts.The trigger condition
When the upstream provider stream is cut mid-generation, the AI SDK emits:
opencode's session processor receives the
finish-stepwithvalue.finishReason === "other"andusage.tokens.output === 0. Pre-fix,the processor accepts that as a legitimate end-of-step.
Symptom (real-world evidence)
I found more than a dozen instances of this exact bug pattern across my own
opencode session database, spanning two providers (
anthropic,openai)and four models (
gpt-5.3-codex,claude-opus-4-6,claude-opus-4-7,claude-haiku-4-5). All exhibit the same shape:Mid-stream cut, not a model decision: in one diagnostic example, the
reasoning text literally ends mid-word —
"...really just wrapping the existing whichlang::detect_language() functi". The upstream stream wassevered before the next chunk arrived.
User-visible behavior pre-fix: the session stores a half-finished
message with no error, no retry, no recovery. In one observed session the
user manually re-prompted ~111s later, succeeded for 3 turns, hit the bug
again, re-prompted again — the "session degradation" pattern users report
in #16214.
The fix
Three small changes:
processor.ts— DetectfinishReason="other"with zero outputtokens on
finish-stepand fail the stream with a retryableAPIErrortagged
metadata.code = "EmptyOther".retry.ts— CapEmptyOtherretries at 3 attempts so a misbehavingprovider can't loop forever. Other retryable classifications keep their
existing unbounded behaviour.
message-v2.ts— Addcase APIError.isInstance(e)tofromErrorthat converts the class instance to its wire form, so the structured
message and metadata reach the TUI instead of being wrapped in a generic
UnknownErrorwhose payload is the JSON-stringified original.Scope: why processor-layer instead of provider-layer
Related #21727 catches a similar truncation pattern at the
@ai-sdk/openai-compatibleprovider'sflush()callback, which works onlyfor OpenAI-compatible providers. This PR catches the same condition one
layer up, in the session processor, where it applies to all AI-SDK
providers — including Anthropic direct, Bedrock, and Vertex. The instances
I observed include Anthropic-direct cases that #21727 cannot reach. The
two PRs are independent and complementary; either order of merge is fine.
How did you verify your code works?
retry.test.ts:EmptyOtheris recognized as retryable.EmptyOther.APIErrorclass instances thrown viaEffect.failround-trip throughfromErrorcorrectly (preservingdata.messageandmetadata.code).bun test test/session/retry.test.ts— 31 pass).bun typecheckadds no new errors.Other user-visible issues this likely helps
symptom class; user has not run diagnostics to confirm mechanism.
satisfies one of four suggested fixes (retry-cap angle for EmptyOther only).
session") —
fromErrorpassthrough makes these errors readable in theTUI; retry cap prevents indefinite loop.
Checklist