test(model-info): cover cache usage observation boundaries - #532
Conversation
|
CI receipt for
This PR changes only the two model-info test files and the research record/index; |
tt-a1i
left a comment
There was a problem hiding this comment.
Reviewed current head 1db7963 after updating to the Effect cohort fix. Standards: no actionable findings. Spec: the added synthetic tests and source-scoped cache usage record distinguish observation from verified causes, and explicitly leave real comparative Session evidence open. This adds no runtime or model behavior. No P0/P1 found. Merge remains conditional on all required current-head CI.
Problem
Refs #156; this does not close the umbrella issue. The diagnostic core already shipped in #372. Remaining evidence needs to distinguish provider field normalization, OpenPI's existing observational contract, and genuine multi-product Session experiments.
Value
Make the current cache observations reviewable and regression-tested without presenting cache misses as verified TTL expiry, billing evidence, or a proven reason for cache invalidation.
Approach
d981de1229ef899957bbe968bc8dcda02a21f477) and a governed research record distinguishing facts, synthetic evidence, and unfinished experiments.Validation
Base: OpenPI
5bf2fe29e52801d79826c2eb573be403f53285e5. Candidate:b409b2b3963f2d017864d1e6e55ac0104c4a7860. Local Windows, Node 24.19.0, Bun 1.3.14, frozen lockfile.Proof: deterministic observation contract
node --test --experimental-strip-types tests/extensions/model-info/cache-diagnostics.test.ts tests/extensions/model-info/index.test.ts tests/extensions/model-info/session-metrics.test.ts bun run check bun run testbun run check: passed (format, lint, typecheck, configuration/discipline checks, Web build).git diff --checkand new local documentation links: passed.bun run test: failed locally, not claimed green. Initial Node group: 1,430 passed, 4 failed, 10 skipped. The four failures were in unchanged git-info/process, setup/integration, shared/setup-config, and workflows/narrator tests (timeouts/Windows file-watcher assertion). The same four files passed 41/41 when rerun serially; that does not establish a unique cause or erase the full-run failure.autoreview --mode local --engine codex --prompt-file <frozen-scope>: clean, no actionable findings. An earlier attempt ended on a usage limit without a verdict; the completed retry reviewed the same four-file candidate.Limits / still open
No provider calls, real OpenPI/Bare Pi/OMP Session recordings, cost/latency experiment, false-positive-rate measurement, or verified TTL cause is represented by these fixtures. Those #156 acceptance items remain open. No user chats, credentials, or raw Session files are published. Upstream CI remains the separate repository-wide validation gate.
Impact