fix(provider): sort Claude tool results by tool use order to preserve prompt cache - #1167
Open
3316891527 wants to merge 2 commits into
Open
fix(provider): sort Claude tool results by tool use order to preserve prompt cache#11673316891527 wants to merge 2 commits into
3316891527 wants to merge 2 commits into
Conversation
… prompt cache When tools execute in parallel, results are emitted to the database in completion order rather than invocation order. When rebuilding prompt history for subsequent turns, ClaudeProvider output tool_result blocks in XML text order. Because Anthropic Prompt Caching requires strict prefix matching, this ordering divergence between turns broke the prompt cache on every subsequent turn following parallel tool calls. - Sort matched tool calls by openToolUses definition order before building Claude content blocks - Expose ClaudeSerializedHistory and buildSerializedHistory for unit testing - Add regression test in ClaudeProviderHistoryTest verifying parallel tool results are ordered by tool_use definition order Fixes AAswordman#1159
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
变更说明 / Description
在 Anthropic 协议开启 Tool Call 模式下,当上一轮包含多个并行工具调用时,用户发送新一条消息后,Claude 的 Prompt Cache 会从该轮工具调用开始的位置整体失效,工具历史全部变成
cache_creation重新计费。本 PR 在
ClaudeProvider构建序列化消息历史时,将匹配到的工具结果(matchedCalls)按当轮tool_use的原始定义顺序进行稳定排序后再组装输出,消除因并行执行异步竞态导致的 XML 文本乱序,恢复 Anthropic 严格前缀缓存命中。背景与动机 / Context and motivation
ToolExecutionManager.executeInvocations会将结果重新按原始调用顺序(Invocation Order)排序放入内存上下文。collector.emit输出并持久化到数据库的(完成顺序 / Completion Order)。parseXmlToolResults解析,导致下一轮请求发往 Claude 的tool_result块顺序与上一轮不一致。tool_result块顺序发生变化,缓存前缀在此处断裂,导致从该位置往后的所有历史 token 产生高额的 Cache Miss。改动范围 / Changes
ClaudeProvider.kt:PromptTurnKind.TOOL_RESULT分支中,调用consumeMatchingToolCalls匹配结果后,根据openToolUses的原始调用顺序(useOrder)对matchedCalls进行排序,确保输出至contentArray的tool_result块与上一轮定义的tool_use顺序严格一致。ClaudeSerializedHistory和buildSerializedHistory的可见性从private调整为internal,以便单元测试覆盖。ClaudeProviderHistoryTest.kt:tool_use顺序。兼容性与风险 / Compatibility and risks
ClaudeProvider在存在多个tool_result时的 JSON 报文块序列化排列,不改变工具执行逻辑或结果文本内容。关联 Issue / Related issue
Fixes #1159
验证方式 / Verification
证据 / Evidence
02ea76a—fix(provider): sort Claude tool results by tool use order to preserve prompt cache检查清单 / Checklist
dev;如目标为main,我已说明这是维护者发布同步 / The target branch isdevfor regular work; if it ismain, I explained why this is a maintainer-led release syncCandidate checks覆盖改动范围,并已区分上游测试源码失败 /Candidate checkscovers the change scope and the upstream test-source failure is identified separately