Make cache child dispatch attributable and stale-safe - #3278
Open
eanz17 wants to merge 1 commit into
Open
Conversation
案例 20(supplier_risk_tier_aggregation)在生产出现一次 8/10 committed failed 且 lastError 为空,收敛到 cache miss 分支派发的动态 child 与 parent 都未完成。 根因未能从现有证据定位——失败 run 的标识未保留,而 cache 与终态判定路径上 的关键判定点没有 Information 级日志,107KB 生产日志里查不到任何 cache 记录。 本提交不猜测根因,只补齐让下次可归因的三处,并修一个已证实的派发缺陷: - CacheModule 的 StepCompletedEvent 分支此前完全没有日志,两处 early return 静默丢弃孤儿 child 完成,waiter 永久失联。现在按 child step marker 区分 「不是 cache child」与「映射丢失」,后者记 Warning;child 完成释放 waiter 时记 Information,与既有 HIT/MISS/PENDING 日志对称。 - 合成的 parent StepCompletedEvent 此前不带 ExecutionId,导致 WorkflowExecutionKernel 的 stale-execution 校验被跳过,超期派发的完成事件 会静默推进 run。现在把 kernel 分配给 parent 的 ExecutionId 存进 waiter 并原样回填。 - child StepRequestEvent 此前不转发 ExternalInvocation。kernel 明确要求派发 子步骤的原语复制该 call site,否则 child_step_type 为 tool_call / connector_call 时子步骤会丢失准入身份。foreach/while 已经这样做,cache 没有。 CacheWaiterState 新增 execution_id 字段(additive,旧状态默认空)。 未包含(另开 issue):cache 尚不支持 sub_param_ 透传,因此需要参数的 child_step_type 仍会静默降级;这属于新增作者面,不在本次可观测性/正确性范围。 验证: - dotnet build src/workflow/Aevatar.Workflow.Core --nologo → 0 error - dotnet test test/Aevatar.Integration.Tests --nologo → 487 passed / 4 skipped (含新增 2 条:call site 转发 + ExecutionId 回填、映射丢失不误完成 parent) - dotnet test test/Aevatar.Workflow.Core.Tests --nologo → 946 passed - bash tools/ci/architecture_guards.sh → passed Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
问题
案例 20(
supplier_risk_tier_aggregation)在生产 Ready 镜像de801ca7上出现一次 8/10 committedfailed,且lastError为空;诊断收敛到 cache miss 分支派发的动态 transform child 与 parent 都未完成。3 分钟后的定向复跑通过(15/15)。根因没有定位,本 PR 也不猜。 障碍是证据不足:失败 run 的 run/actor/动态 child 标识未保留,而 cache 与终态判定路径上的关键判定点没有 Information 级日志——107KB 生产日志里查不到任何 cache 记录。8 条候选机制经独立对抗核实全部被推翻(它们只能产生"挂住",产生不了 committed failed)。
本 PR 只做两件事:让下次同类失败可归因,并修一个已证实的派发缺陷。
方案
src/workflow/Aevatar.Workflow.Core/Modules/CacheModule.csStepCompletedEvent分支此前完全没有 Logger 调用,两处 early return 静默丢弃孤儿 child 完成,waiter 永久失联。现在用 child step marker 区分「不是 cache child」(保持安静,绝大多数完成都属此类)与「映射丢失」(记 Warning);child 完成释放 waiter 时记 Information,与既有 HIT/MISS/PENDING 日志对称。StepCompletedEvent此前不带ExecutionId,WorkflowExecutionKernel的 stale-execution 校验因此被跳过,超期派发的完成事件会静默推进 run。现在把 kernel 分配给 parent 的ExecutionId存进 waiter 并原样回填。StepRequestEvent此前不转发ExternalInvocation。kernel 明确要求派发子步骤的原语复制该 call site,否则child_step_type为tool_call/connector_call时子步骤会丢失准入身份。ForEachModule/WhileModule已经这样做,cache 没有。workflow_state.proto:CacheWaiterState新增execution_id(additive,旧状态默认空)。影响路径
只影响
cache原语的 miss 分支。hit / pending-join 路径未改动。不包含
cache 尚不支持
sub_param_透传,因此需要参数的child_step_type仍会静默降级为默认行为(transform 的op=identity)。这属于新增作者面,不在本次可观测性/正确性范围,另开 issue 跟进。验证
dotnet build src/workflow/Aevatar.Workflow.Core --nologodotnet test test/Aevatar.Integration.Tests --nologodotnet test test/Aevatar.Workflow.Core.Tests --nologobash tools/ci/architecture_guards.sh新增 2 条测试:
CacheModule_OnMiss_ShouldForwardCallSiteAndEchoParentExecutionIdOnCompletion、CacheModule_WhenChildMappingIsLost_ShouldNotCompleteAnyParent。🤖 Generated with Claude Code