fix(provider): strip reasoning blocks from non-latest assistant messages to fix compaction#26892
Open
jralph wants to merge 1 commit into
Open
fix(provider): strip reasoning blocks from non-latest assistant messages to fix compaction#26892jralph wants to merge 1 commit into
jralph wants to merge 1 commit into
Conversation
…ges to fix compaction The Anthropic/Bedrock API requires thinking blocks in the latest assistant message to be replayed exactly as originally returned. Signatures on historical thinking blocks can become invalid due to text modifications during stream processing, causing compaction and multi-turn conversations to fail with 'thinking blocks cannot be modified'. Strip reasoning blocks from all non-latest assistant messages before sending to the API. The API only validates the latest assistant message, so historical thinking blocks are safely removable. Additional fixes: - Skip normalizeMessages empty-content filtering for assistant messages - Skip sanitizeSurrogates on signed reasoning parts - Detect Bedrock signatures in hasSignedReasoning (was only checking anthropic) Fixes anomalyco#14332
jralph
force-pushed
the
fix/strip-reasoning-blocks-compaction
branch
from
2610bd3 to
4bd306b
Compare
Contributor
|
The following comment was made by an LLM, it may be inaccurate: Based on the search results, I found a potentially related PR: Related PRPR #14393: "fix: preserve thinking block signatures and fix compaction headroom asymmetry" Why it may be related:
This appears to be a closely related fix to the same compaction problem space. You may want to check if PR #14393 is still open and whether it addresses the same |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Issue for this PR
Closes #14332
Type of change
What does this PR do?
Strips reasoning blocks from non-latest assistant messages before sending to the API. Both Anthropic and AWS
Bedrock document that thinking blocks from prior turns can be omitted, the API only validates signatures on the latest assistant message.
This avoids the
thinking blocks cannot be modifiederror that occurs when historical signatures become invalid during compaction or multi-turn replay.Also fixes
hasSignedReasoninginmessage-v2.tsto detect Bedrock signatures (was only checking theanthropickey), and skipssanitizeSurrogates/empty-content filtering on assistant messages to protect the latest message's thinking blocks. It was failing on large sessions consistently and reliably.How did you verify your code works?
226 tests pass (6 new), 0 failures. Manually tested compaction with Opus 4.7 on Bedrock, works where it previously crashed. It wasn't previously failing on small context sessions (for example, saying "hi" and compacting worked), but was failing on larger 180k+ context sessions, and is not anymore.
Screenshots / recordings
Original Issue:
Checklist