Repository navigation
test(qa): seed and grade the prescribed worktree tree composition - #201
qiyuanhuakai wants to merge 5 commits into
Conversation
|
AI 辅助源码审查(dot;中文顶层评论,非 APPROVE / REQUEST_CHANGES)。 审查 head:45d30200fdd3ee74d8b058dae36f267dee45213e
验证与继承边界:本次重新核对该SHA的两文件及公式/调用顺序,没有新跑完整浏览器性能测试或重启恢复实验,也不把历史探针冒充本轮新执行。作者8/8及类型检查声明不作为我的独立结果。CI 39109829751 当前completed/failure,Validate在Lint/type-check失败;既有日志的TS7016属于上游ports声明缺口,不是上述benchmark问题的复现证据,也不重复计作本PR新问题。下游#205补声明不能修复此处评分/采样逻辑。本审查不保证没有其他缺陷。 |
Review findings on the task-44/50 performance driver and benchmark: 1. A failed operation could still pass. cacheOpenProbe recorded a duration even when it found no draft, cacheExportBatch recorded a duration and a null digest after catching, and event-visible returned seen=false with a fast duration — while openProbeFound / selectedPageDigests / eventVisible.seen never entered any scoring condition, so a run passed on latency alone even when open/export failed or the event was invisible. The three rows (interaction, selected-page, event-visible) now require the operation to have actually succeeded, with the observed success counts carried in each row's note. No threshold was raised and no check removed; a green run still passes. 2. Coverage rounding. The drafts coverage row read `coveragePct === 100` and the percentage is rounded to two decimals, so at the 100k scale five missing draft headers (99996/100001) rounded to 100.00 and passed. The EXACT visited count now decides for both rows (pins keeps its digest guard and gains the count); the rounded figure stays as display-only. A missing dimension or a missed hash in a load profile fails its own row instead of being dropped from the table. 3. Network conditions. runDraftsLeg resets the emulated network to unlimited at its end, so the network100ms profile's formal samples measured an unlimited network while the report still labelled them 100ms-RTT. The driver now re-applies the profile's conditions after the recovery legs and VERIFIES them with a real measured round trip (an emulated RTT cannot be beaten); a failed verification throws instead of labelling unlimited samples as remote. 4. Restart. The recovery row was named "offline/restart/switch/resume" while only three legs were proven: the offline read-back ran with the store/cache still open and the dispose/close happened only AFTER the read, so no restart was ever executed. runDraftsLeg now destroys the cache, the store AND the page, rebuilds from the same origin, cuts the network at CDP and reads the draft back, and the restart is a graded leg of its own (four legs, frozen literal). 5. Load coverage. The draft leg only ever imported 2KiB single-body-part drafts, so large content and attachments had no coverage. A new loads leg imports the frozen LOAD_PROFILES (a 512KiB single body; a 64KiB body plus two 128KiB attachments), verifies every part's stored and read-back sha256 against the independent Node-side oracle, then destroys/rebuilds the world and reads every part back offline. Results are reported per load type, never pooled with the 2KiB drafts, and the drafts coverage row's scope now says so explicitly. Verified: the row builder is driven by app/runtimeV090Performance.test.ts (15 tests, RED before the fix: a failed-but-fast run passed all three rows, the rounded coverage passed with records missing, and the load dimension was ungraded). The two new benchmark legs were also exercised end-to-end against a real Vite + headless Chromium + IndexedDB + CDP: drafts-leg restart match true, loads 4/4 and 12/12 parts verified with 4/4 and 12/12 read back after the offline restart (restartOnline false).
|
AI 辅助增量源码复核(dot;中文顶层评论,非 APPROVE / REQUEST_CHANGES)。 审查 head:33af688b9abbaa103aa0976eae8da119a5dae7bc 原四项评分/采样问题已有实质修复,但新增两个阻塞,不能判整体验收通过。
新问题
原负载覆盖缺口:部分补齐,尚不能关闭全部范围。 独立验证与继承边界 本次 3 文件增量没有整合 #200 后续 App delivery 修复;其余 stack 问题仍按各自 SHA 判断。CI 终态补记(2026-10-19 04:07 UTC):CI 39197361222已结束,Validate failure(19 个新增测试类型错误 + 6 个继承 ports 错误),其余 11 个 Electron/Bridge installer 矩阵任务 success,Complete CI failure,Publish draft release skipped。打包成功不代表 benchmark 已执行或性能通过。本审查不保证不存在其他缺陷。 |
The exact-count draft coverage row added last round was itself failing a
HEALTHY run. The drafts leg and the loads leg write into the same
DraftStore + DraftCache IndexedDB, and the load leg built its contexts as
`${prefix}-load-<label>-NNNNNN`, which starts with `${prefix}-`. The plain
draft coverage enumeration filtered on that PREFIX, so the 8 load headers
were counted as 2KiB single-body-part drafts and the exact count read
10009 against the frozen 10001 (10k) and 100009 against 100001 (100k) —
reproduced with the unmodified runLoadsLeg + runCoverage over controlled
responses, at both scales.
The plain dataset now has a STRICT identity instead of a prefix test: a
context counts only when it is the fixture's `${prefix}-NNNNNN` encoding.
The load leg keeps its own namespace (`${prefix}-load-<label>`) through a
shared helper so the two cannot drift, and it is still fully graded — every
part is imported, hash-verified and read back offline after a real restart,
and an unmeasured load profile still emits a FAILING row. No threshold was
moved and no assertion was dropped; each dataset is simply counted against
its own frozen expectation instead of being pooled behind a shared prefix.
The hand-built COVERAGE objects in the existing tests could not see this
integration conflict, so the regression drives the REAL runDraftsLeg,
runLoadsLeg and runCoverage through a controlled page/store double and
requires a healthy run to score healthy. RED before the fix:
"expected 10009 to be 10001"; GREEN after.
The new test file also failed the project type check with 19 diagnostics
(TS7016 x2 for the undeclared .mjs modules, TS7006 x15 for untyped
callbacks, TS2339 x2 for the inferred `{}` options bag). The two engine
modules now carry .d.mts declarations and the test is fully typed, with no
@ts-ignore, no `as any` and no implicit any. vue-tsc reports zero
diagnostics for this file; the six inherited collaborationPorts/threadPorts
TS7016 errors belong to PR #200's own head and are untouched here.
|
AI 辅助增量源码复核(dot;中文顶层评论,非 APPROVE / REQUEST_CHANGES)。 审查 head:5e1ebc0ef1ecab1f6eac9504ec514e80baed9923。 上轮两个 P1 已修复。
新增联合回归 L391–405确实把真实 drafts→loads→coverage 编排放在同一受控 header 集合上;它覆盖此前手工 COVERAGE 对象看不到的冲突。 仍需保留原负载覆盖限制。 验证、继承与 CI 边界 当前 #201 head 仍从 ccca97b 分叉;#200 最新 a828d34 不在其祖先链。CI 检出的实际测试合并提交 7612900 则已合入该最新 #200,所以旧六处 ports 错误在本次 CI 合并树消失。不能继续把它们列为当前 CI 红灯,也不能反过来说 #201 head 已回补所有 #200 修复。 |
This branch's App.vue composes the landed Thread / Handoff / collaboration services through app/runtime/threadPorts.ts and app/runtime/collaborationPorts.ts, but only the .js modules were inherited: the six sibling .d.ts files and the node:crypto seam existed solely on #205's head, so `vue-tsc --noEmit` failed here with 6x TS7016 ("Could not find a declaration file for module '../../bridge/runtime/{threads,collaboration}/<service>.js'"). Bring the actual bytes from #205's head 32ca6b7 rather than re-deriving them: - bridge/runtime/threads/checkpointService.d.ts, handoffService.d.ts - bridge/runtime/collaboration/coordinator.d.ts, delegationService.d.ts, fanoutService.d.ts, mailbox.d.ts - app/utils/nodeCrypto.ts (+ its test) and the vite.config.ts alias that maps `node:crypto` onto it, so a browser realm gets the browser primitives while a Node realm (bridge, SEA, workers, tests) keeps the real builtin RED: 6 TS7016 at app/runtime/{threadPorts,collaborationPorts}.ts. GREEN: vue-tsc --noEmit exit 0, vite build exit 0, nodeCrypto.test.ts 4 green.
The reviewer's remaining gap: LOAD_PROFILES still imported scaled-down stand-ins (512 KiB body, two 128 KiB attachments) while the frozen generation input app/dev/runtime-v090/fixtures/seed090.json prescribes 1 MiB attachments on every 100th draft, a 16 MiB body and one 64 MiB large attachment — so the prescribed scales themselves were never exercised through the real store. Carry the literals: `large-body` is now a 16 MiB body, `multi-part-attachment` is the every-100th record's shape (2 KiB body plus two 1 MiB attachments), and a new `oversized-draft` profile IS the fixture's single extra draft (16 MiB body + 64 MiB attachment). `expectedPartHash` is memoized: the oracle's INPUTS and OUTPUT are unchanged, only the re-generation of an identical multi-MiB buffer is avoided. The acceptance boundary is explicit and checked, not asserted. The every-100th DISTRIBUTION over the whole emitted dataset (100 such drafts at 10k, 1000 at 100k) is verified by scripts/qa/runtime-v090-fixtures.mjs `check`, executed at both scales by app/runtimeFixture.test.ts; this leg SAMPLES the prescribed sizes (four drafts per sampled class plus the one prescribed oversized draft) because importing every 100k attachment draft (1000 x 1 MiB) is outside this leg's budget. What is graded here is the prescribed SIZE of each load class through the real store, hash-verified per part against the independent Node-side oracle, on import AND after a real offline restart — recorded per part in load-import-hash-ledger.json. RED: the frozen-profiles assertion fails on the old stand-in sizes. GREEN: app/runtimeV090Performance.test.ts 21/21 (the profiles now assert 16 MiB / 2 KiB + 1 MiB / 16 MiB + 64 MiB exactly, and the load header count derives from LOAD_PROFILES instead of a frozen 8); app/runtimeFixture.test.ts 8/8 including the 100k whole-dataset distribution check. oxlint exit 0; vue-tsc --noEmit exit 0.
|
本轮推送 2 个提交,PR head 已从 5e1ebc0 推进到 930bd4c。 1. 大负载覆盖缺口:已补齐,并明确缩小验收声明 LOAD_PROFILES 此前用的是缩小替代规格(512 KiB body、两条 128 KiB attachment),未触及 seed090.json 冻结输入规定的规模。现按冻结输入字面值执行:
验收边界由检查保证,不是只在注释里声明:
本地验证: 2. #200 修复回补情况(精确表述) 精确说法:六份服务声明与 node:crypto 接缝在本 head 上以本分支自己的提交 同时如实说明其余部分: 3. 本地全量测试结果 vitest run:620 个测试文件中 615 个通过;2 个文件失败( |
|
AI 辅助独立增量复核(dot)。 审查 head:930bd4cc69950be6c64f1e95a1bf0de78cb9d637。相对 5e1ebc0 为 2 个提交、11 文件;发布前重读 head/comments,PR open、未合并。 结论:原“大负载尺寸没有体现”的覆盖缺口已补齐,验收范围也已明确缩小为采样尺寸覆盖;本轮未确认新的增量缺陷。不能由此认定完整分布的实际性能已验收。 LOAD_PROFILES L144–170现在包含 4 条 16MiB 正文、4 条 2KiB 正文加两份 1MiB 附件、1 条 16MiB 正文加64MiB附件。尺寸对应 seed090 冻结输入 L7–11,没有继续用512KiB/128KiB替代大文件。已有 runLoadsLeg按各 profile 逐 part 写入与核 hash,driver L232–246遍历当前 LOAD_PROFILES,为新增 oversized-draft 单独评分;未测量的 profile 不会获得通过。hash memoization 以 size/seed 为 key,只复用预期摘要,不将存储回读结果拿来充当 oracle。 独立定向验证:重新取回当前实际 benchmark/driver/测试,以 Node 24 内置类型擦除和 node:assert 适配器执行测试文件的 21 个原始回调,21/21通过。适配只替换 Vitest 导入/断言及临时输出位置;测试使用受控 page/store 响应,实际编排与评分函数未改。新增尺寸断言、第三类负载、9个负载header与普通draft覆盖隔离均通过。此结果不等于运行了真实浏览器、IndexedDB或完整Vitest,也没有测出真实时间、网络、峰值内存或原生升级性能。 仍需保留的验收边界:
堆叠修复:新增六份声明、nodeCrypto seam/测试及Vite alias共9文件,与已审 #200 a828d34 逐一blob SHA一致,已实际回补本head;没有把此前App接线/协作回执等其他#200修改一并算作已回补。上轮两个P1修复维持原结论。 CI终态更新(2026-10-19 14:04 UTC):head未变,39264057723 completed/success。Validate、11个安装/打包job、Complete CI共13个job成功,发布跳过。Validate日志显示 runtimeV090Performance.test.ts 21/21、runtimeFixture.test.ts 8/8,整套618 files / 6206 tests passed,2 files / 5 tests skipped;lint/type-check和web build成功。实际测试合并提交00bccf988b212d6ce7edf9df3969e24103b05c71合入#200 a828d34。CI属于外部运行证据,仍不能替代实际性能benchmark或全量附件分布导入验收。 |
Task 50 — benchmark composition seeding.
runtime-v090-benchmark.mjsgainsSLO,DATASET,TREE,planComposition,seedTreeCompositionandplanCompositionAgreesWithFrozenFixtureOracle;runtime-v090-performance.mjsis the driver. The two files are import-coupled (the driver importsSLO/DATASET/TREEfrom the benchmark) so they cannot be separated into different commits — the driver's own header still says "Task 44 — performance DRIVER".Verification:
app/runtimeFixture.test.ts(the referencing test) 8/8 green;vue-tsc --noEmitexit 0; oxlint warnings-only.