Skip to content

chore(ci): trace Windows cold command startup - #363

Closed
vincentkoc wants to merge 11 commits into
feat/plugin-resource-tokenjuice-20260923from
chore/windows-startup-diagnostics-20260923
Closed

vincentkoc wants to merge 11 commits into
feat/plugin-resource-tokenjuice-20260923from
chore/windows-startup-diagnostics-20260923

Conversation

@vincentkoc

@vincentkoc vincentkoc commented Sep 22, 2026 •

Copy link
Copy Markdown
Member

Related: #305

What Problem This Solves

Windows fixture commands can exhaust startup and cleanup budgets before a plugin loads. This draft records bounded producer receipts and now tests a controller-environment candidate. It does not claim a proven Windows startup repair.

User Impact

The PowerShell controller now resolves SystemRoot case-insensitively from its parent environment and inherits that parent environment. The native target still receives the caller's exact serialized environment. Target PATH/COMSPEC resolution, cwd, parser, command order, startup 10s/cleanup 2s budgets, admission, and cleanup guards remain unchanged. No retry or prewarming is added.

Why This Change Was Made

Stacked on #362 at 42057057a87c84e909fb84c06c2baa4a5b941fc1; current candidate head 6eedbb511a1c800e1887c9f58f5fcc49b5aebd46.

The controller previously received the target's restricted environment. That crosses the ownership boundary between the trusted helper runtime and the command it admits. The candidate separates those inputs at helper launch while leaving the native target block unchanged. Whether this explains the observed Windows failures remains pending native proof.

The earlier module-free observer is retained: fixed JSON formatting and .NET file APIs avoid diagnostic cmdlet autoload before pipe connection. All product cmdlets and event sites remain intact. Separate parent, Worker, PowerShell, and native receipts preserve the earliest Inspector attempt and independent Tokenjuice archive/repository attempts outside disposable fixture roots.

Each producer is capped at 32 records, 2 KiB per record, and 64 KiB total. Records exclude arguments, environment values, command output, and arbitrary exception messages.

Evidence

  • Original Tokenjuice required Windows and advisory Windows failed before plugin execution. Their cleanup-unconfirmed outcomes remain unresolved.
  • At 950f1d8d646b04f85d545003e45f69e9d7710c47, all four Tokenjuice attempts recorded only helper-script-entered, then EOWNERSTART; helper SIGTERM closure and terminal receipts were observed with cleanupError=null. The earliest required Inspector attempt stalled after compile-begin and retained EOWNERCLEANUP. Later successful Inspector attempts do not erase that failure.
  • Required 950 receipt archive SHA256: 02fb361e7106500e94de93e1bb11cb6fd07502c267b9fb40a277a0739731b707; advisory: 730343f13c573067786ce60a16b044a74dd65b51a5c0fabdbf6242f156d6bb21.
  • At b7f6606319d8927124061f55b8ec015c0e671b4f, both required Windows and advisory Windows failed on Node v24.20.0. All four Tokenjuice attempts reached helper-entered, pipe connection, request read, and parse-begin (78–82 ms), but no parse-end, owner monitor, native admission, or READY. All ended EOWNERSTART plus EOWNERCLEANUP without terminal cleanup receipts. This locates the original parser boundary; it does not establish the cause.
  • Required b7f archive/repository and advisory repository recorded helper kill/control closure and a parent absence probe, but still lacked authoritative terminal cleanup. Advisory archive recorded cleanup-deadline-fired without a helper-kill record, and its final parent probe still reported absence=false at 14,025 ms. That helper closure remains unconfirmed. Inspector succeeded with full closure in both runs (READY 8,730 ms required; 4,470 ms advisory).
  • Required b7f receipt archive SHA256: b449b272f49e1273b00a04c94690fbc90a4cdd424d5fca508a6b4bbfc9c878cb; advisory: cd692f85c81699425418e735120474aa18202d5139a1aba9d419c6dec6a298ad. All 20 bounded startup files were retained and individually hash-verified.
  • Current candidate: node --test test/owned-command.test.mjs test/owned-command-startup-diagnostics.test.mjs passed 16 tests with 13 platform skips in 24.148s. Syntax and git diff --check passed. The new Windows regression is skipped locally; these results do not validate native Windows execution.
  • Independent actual-diff review is clean after correcting the new test's teardown ordering. One teardown awaits Worker termination before removal and retains the fixture when helper/native cleanup is unknown.
  • Increment since b7f: production +4/-2 (two explanatory lines; two executable substitutions), tests +75/-0. Full PR: +622/-11, 633 changed lines across seven files. Owner code +357/-11; tests +224/-0; docs +41/-0. Positive owner growth supplies otherwise unavailable cross-process startup evidence; the environment candidate adds no new framework or dependency.

Fixture Impact

Existing fixtures and pins are unchanged. The original restricted Tokenjuice environment builder and serial command order are unchanged. The new Windows copied-owner regression uses .NET-only synthetic comparisons in the actual helper, then runs a real native child: parent-only data must stay out of the target, and target conflicting/target-only values must survive. Receipts contain booleans, not real environment values.

Fixture Verification

The naturally triggered Default Track workflows must validate the new Windows regression and the original serial restricted Tokenjuice archive/repository path. Required evidence is parse-end, owner monitor, native admission/READY, and authoritative cleanup, with the earliest attempts preserved. Native proof is pending at publication. Any later pass does not retroactively close the earlier uninstrumented or cleanup-unconfirmed failures.

Notes

Diagnostic I/O and compilation affect timing. Missing records remain unavailable evidence; progress messages, Worker termination, and PIDs are not native cleanup receipts. This remains a diagnostic draft with a causal candidate. Fresh exact-head native autoreview and Windows evidence are required before any merge decision.

@clawsweeper

clawsweeper Bot commented Sep 22, 2026 •

Copy link
Copy Markdown
Contributor

🦞👀
ClawSweeper picked this up.

Pull request received. I will update this pull request when review starts.

ClawSweeper review complete

ClawSweeper finished reviewing this revision. The review result is being finalized.

View the workflow run.

@clawsweeper clawsweeper Bot added P2 Normal priority bug or improvement with limited blast radius. merge-risk: 🚨 automation 🚨 Merging this PR could break CI, automerge, proof capture, label sync, or automation. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. labels Sep 22, 2026
@clawsweeper

clawsweeper Bot commented Sep 22, 2026 •

Copy link
Copy Markdown
Contributor

Codex review: blocked before merge. Reviewed October 8, 2026, 7:27 AM ET / 11:27 UTC (Revision 7).

ClawSweeper review

What this changes

This PR adds bounded Windows startup receipt files and separates controller and target environments, alongside stacked plugin resource-workload reporting and scenarios.

Example: A Windows fixture supplies a restricted command environment

  • Before: The PowerShell controller receives that restricted environment and can time out before admitting the command.
  • After: The controller inherits its parent's environment while the native command receives the caller's serialized environment, with separate bounded startup receipts.

Review scores

Measure Result What it means
Overall readiness 🦐 gold shrimp (3/6) Useful native diagnostic evidence supports the narrow investigation, while a real Windows fixture defect and unresolved stack integration limit readiness.
Proof confidence 🐚 platinum hermit (4/6) Sufficient (logs): Recorded evidence in the merged extraction ties this exact head's real Windows cold fixture flow to READY and authoritative terminal cleanup for the first Inspector, tar and Git attempts. That satisfies the diagnostic startup/cleanup claim, despite nonzero fixture exits; it does not prove the intermittent failure fixed or the stacked Tokenjuice workload qualified. Raw-log re-audit was blocked by the reviewer network boundary. Receipt compatibility is additive: v1 reading remains supported and no existing files are migrated.
Patch quality 🦐 gold shrimp (3/6) 2 actionable review findings remain.

Product

Kind: Maintenance · Worth it: Yes · Fix scope: Partial
User problem: Windows fixture checks can time out before a plugin loads and leave command cleanup uncertain.
Reason: The area owner's bounded observation provides useful evidence for a real recorded failure. Its stated temporary investigation scope remains valid, but it does not establish a complete startup repair.

Merge readiness

⛔ Blocked before merge - 4 items remain

Keep this maintainer-authored diagnostic draft open. The controller-environment repair has landed separately, but independent receipt collection remains distinct; the conflicted stack and two concrete defects prevent landing this head.

Priority: P2
Reviewed head: 6eedbb511a1c800e1887c9f58f5fcc49b5aebd46

Before merge

  • Use an ordinary empty file for Git's global configuration (P2) - On Windows, prepareRepository() sets GIT_CONFIG_GLOBAL to Node's null-device path before its first git init. The native discriminator recorded in feat(profiling): exercise Tokenjuice agent tool results #362 shows this input makes Git exit 128; substituting an existing empty ordinary file makes initialization succeed. This introduced adapter is also exercised by the default test suite, so successful supervisor startup still leaves the fixture failing. Incorporate the repair through the stacked parent.
  • Describe the controller environment change in the release note (P3) - The entry still claims that environment policy is unchanged, but this head switches every Windows helper launch from the caller environment to the parent environment and changes SystemRoot selection. Preserve the previous review's correction: distinguish the changed controller inputs from the unchanged native target environment so the release note accurately describes the compatibility boundary.
  • Resolve merge risk (P1) - The conflicted stack has no reviewed merge result incorporating the landed environment repair and newer startup-phase owner; its integrated Windows command behavior remains unverified.
  • Complete next step (P2) - Refresh the conflicted stack against its current parent and current main before requesting landing review.

Findings

  • [P2] Use an ordinary empty file for Git's global configuration — scripts/resource-workloads/tokenjuice.mjs:83-85
  • [P3] Describe the controller environment change in the release note — CHANGELOG.md:5

Tests

  • Low-value test test/resource-workload-tokenjuice.test.mjs: Tokenjuice scenario declares matched OpenAI controls and fixed one plus five operations: Pins the manifest's current scenario defaults rather than exercising workload behavior; remove this redundant assertion.
  • Low-value test test/resource-workload-tokenjuice.test.mjs: agent cycle uses native admission, matching wait, fresh session and terminal reply: Asserts an exact mocked RPC call sequence and synthetic completion callbacks; retain failure-outcome coverage instead of this implementation-mirroring test.
  • Missing end-to-end proof: The integrated branch after conflict resolution has not been exercised. Successful isolated Tokenjuice matched-pair execution remains owned by the stacked workload PR; the recorded Windows startup and cleanup observations do not prove that workload. No reviewer-run base/head regression was executed.
Agent review details

How this fits together

Crabpot checks OpenClaw plugin compatibility and resource behavior. Its command supervisor starts fixture and inspector commands, enforces deadlines, and reports whether their processes and output streams closed.

flowchart TD
  A[Fixture or inspector command] --> B[Command supervisor]
  C[Parent environment] --> D[PowerShell controller]
  B --> D
  E[Caller command environment] --> F[Native command in Windows Job]
  D --> F
  F --> G[Readiness and cleanup results]
  B --> H[Bounded startup receipt files]
  D --> H
  F --> H
Loading

Technical review

Best possible solution:

Keep the landed controller repair and shared phase diagnostics, preserve the historical receipts, and retain additional temporary instrumentation only within the already-recorded investigation scope.

Do we have a high-confidence way to reproduce the issue?

No high-confidence deterministic reproduction of the intermittent cold-start failure is established; recorded Windows failures and phase receipts locate observed boundaries without proving a common cause.

Is this the best way to solve the issue?

Partly: independent bounded receipts served the investigation, but the environment repair and normal startup-phase diagnostics now have landed owners, and this conflicted temporary stack is not a production-ready solution.

Full review comments:

  • [P2] Use an ordinary empty file for Git's global configuration — scripts/resource-workloads/tokenjuice.mjs:83-85
    On Windows, prepareRepository() sets GIT_CONFIG_GLOBAL to Node's null-device path before its first git init. The native discriminator recorded in feat(profiling): exercise Tokenjuice agent tool results #362 shows this input makes Git exit 128; substituting an existing empty ordinary file makes initialization succeed. This introduced adapter is also exercised by the default test suite, so successful supervisor startup still leaves the fixture failing. Incorporate the repair through the stacked parent.
    Confidence: 0.98
  • [P3] Describe the controller environment change in the release note — CHANGELOG.md:5
    The entry still claims that environment policy is unchanged, but this head switches every Windows helper launch from the caller environment to the parent environment and changes SystemRoot selection. Preserve the previous review's correction: distinguish the changed controller inputs from the unchanged native target environment so the release note accurately describes the compatibility boundary.
    Confidence: 1

Overall correctness: patch is incorrect
Overall confidence: 0.94

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning medium; reviewed against 4ff73eb1397d.

Merge-risk options

Maintainer options:

  1. Reconcile the diagnostic stack (recommended)
    Refresh the branch against its current parent and main, preserve the landed command-owner behavior, and review the resulting Windows integration.
  2. Retire the observation branch
    Preserve its receipts as investigation evidence and let the landed repairs and remaining workload PR carry the production work.

Provenance checked

  • scripts/owned-command.mjs: controller environment changes intended behavior with a stated reason (08223ad: The original PR created one bounded command owner shared by inspector callers, with exact target environment forwarding and no uncontained fallback.)
  • scripts/owned-command.mjs: startup and cleanup observation keeps the original intent (53258a3: The audit extension preserves first command failure and reports cleanup uncertainty rather than treating probe errors as extinction.)
  • scripts/owned-command-windows.ps1 keeps the original intent (08223ad: A compiler-free bootstrap contains the helper and compiler children before native command admission.)
  • scripts/owned-command-windows.cs keeps the original intent (08223ad: Windows commands enter a private Job atomically and successful cleanup requires observed extinction.)
  • scripts/resource-coverage.mjs keeps the original intent (60ad5b2: Report source-inventory coverage separately from fixtures, validate producer identity and retain partial failures without inventing workload credit.)
  • scripts/generate-report.mjs and scripts/report-lib.mjs keeps the original intent (60ad5b2: Optional resource inputs augment the existing report without changing default compatibility results or performance gates.)

Testing

Proof path: shipped entry point. Added test files: 7.

Security

None.

Evidence

What I checked:

  • Repository policy applied: Read the complete root AGENTS.md. No nested AGENTS.md or maintainer-notes directory was found; fixture-driven configuration, credential-free defaults, and external-source boundaries informed the review. (AGENTS.md:1, 6eedbb511a1c)
  • Pinned introduced changes: The verified merge-base-to-head delta contains 22 files, 2,745 additions and 31 deletions, including stacked workload changes. Base-only workflows and reports were not attributed to this PR. (6eedbb511a1c)
  • Environment repair already on main: Current main resolves SystemRoot from the parent environment and launches the controller with process.env while serializing options.env for the native target. GitHub confirms fix: preserve the Windows command controller environment #374 merged the extracted repair. (scripts/owned-command.mjs:588, 4ff73eb1397d)
  • Merged repair provenance: The extracted environment repair appears in commit 6e78c4d, dated September 26, 2026. Local branch containment places it on main; no containing release tag was returned. (scripts/owned-command.mjs, 6e78c4df41dc)
  • Newer shipped-main diagnostic path: Current main collects fixed startup phases through the existing control pipe and Worker port. fix(ci): report command startup phases on failure #400 merged this implementation; it does not provide this draft's independent per-producer files. (scripts/owned-command.mjs:16, 8b984c058387)
  • Exact-head native observations: The recorded evidence in fix: preserve the Windows command controller environment #374 reports this exact diagnostic head reaching READY and authoritative terminal cleanup for the earliest Inspector, tar and Git attempts in required and advisory Windows runs. Tar exited 2 and Git exited 128, so these observations prove startup and cleanup rather than successful fixture preparation or resolution of the historical intermittent failure. (6eedbb511a1c)

Likely related people:

  • vincentkoc: Suggested for follow-up; no historical authorship or introduction is verified. (role: unverified routing candidate; confidence: low)
  • Sebastien Tardif: Suggested for follow-up; no historical authorship or introduction is verified. (role: unverified routing candidate; confidence: low)

Review metrics

Metric Value Why it matters
Introduced code growth production +1,429/-23; tests +1,315/-6; documentation +201/-0 The pinned delta includes substantial stacked workload work, so the seven-file diagnostic count in the body does not describe the full review surface.

Root-cause cluster

Relationship: partial_overlap
Canonical: #305
Summary: This draft investigates the canonical Windows startup report; related work does not establish that every observed failure has the same cause.

Members:

Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything.

Labels

Label changes:

  • add proof: sufficient: Contributor real behavior proof is sufficient.
  • add status: ⏳ waiting on author: ClawSweeper has contributor-facing work open and is waiting for author action.
  • remove status: 👀 ready for maintainer look: Current PR status label is status: ⏳ waiting on author.

Label justifications:

  • P2: This is a bounded investigation of Windows fixture startup and cleanup failures, without evidence of an urgent end-user runtime regression.
  • merge-risk: 🚨 automation: The diff changes command supervision used by CI, and its conflicted integration with the newer owner remains unverified; the temporary observer's timing cost is already explicitly acknowledged.
  • rating: 🦐 gold shrimp: Overall readiness is 🦐 gold shrimp; proof is 🐚 platinum hermit and patch quality is 🦐 gold shrimp.
  • status: ⏳ waiting on author: ClawSweeper has contributor-facing work open and is waiting for author action.
  • proof: sufficient: Contributor real behavior proof is sufficient.

Rating scale

6/6 🦀 challenger crab · 5/6 🦞 diamond lobster · 4/6 🐚 platinum hermit · 3/6 🦐 gold shrimp · 2/6 🦪 silver shellfish · 1/6 🧂 unranked krab. Overall follows the weaker of proof and patch quality; ✨ marks media proof (a screenshot, video, or linked artifact) that directly shows the changed behavior.

Workflow

ClawSweeper edits this one comment on every review. Comment @clawsweeper re-review for a fresh review only; repair and merge need explicit maintainer commands such as @clawsweeper autofix or @clawsweeper automerge.

History

Review history (6 earlier review cycles)
  • reviewed 2026-09-22T18:23:00.385Z sha 439760b :: blocked before merge. :: none
  • reviewed 2026-09-22T19:01:21.060Z sha 8b8bc48 :: needs changes before merge. :: none
  • reviewed 2026-09-22T19:29:24.686Z sha 950f1d8 :: blocked before merge. :: none
  • reviewed 2026-09-22T19:54:42.779Z sha b7f6606 :: blocked before merge. :: none
  • reviewed 2026-09-22T20:26:12.637Z sha 6eedbb5 :: blocked before merge. :: [P3] Describe the controller environment change in the release note
  • reviewed 2026-09-22T20:50:17.304Z sha 6eedbb5 :: blocked before merge. :: [P3] Describe the controller environment change in the release note

@vincentkoc
vincentkoc force-pushed the chore/windows-startup-diagnostics-20260923 branch from 439760b to 8b8bc48 Compare September 22, 2026 18:53
@vincentkoc
vincentkoc changed the base branch from fix/plugin-resource-missing-receipts-20260923 to feat/plugin-resource-tokenjuice-20260923 September 22, 2026 18:55
@vincentkoc vincentkoc closed this Sep 22, 2026
@vincentkoc vincentkoc reopened this Sep 22, 2026
@vincentkoc

Copy link
Copy Markdown
Member Author

@clawsweeper re-review exact head 6eedbb5. Diagnostic draft plus controller-environment candidate; native Windows qualification is pending, with no merge or proven repair claim.

@clawsweeper

clawsweeper Bot commented Sep 22, 2026

Copy link
Copy Markdown
Contributor

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event exact_review_queue).
Result: when the review finishes, ClawSweeper will create the durable review comment if needed or update the existing comment in place.

@clawsweeper

clawsweeper Bot commented Sep 22, 2026 •

Copy link
Copy Markdown
Contributor

🦞👀
Exact review queued.

Re-review progress:

@clawsweeper clawsweeper Bot added rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. and removed rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. labels Sep 22, 2026
@vincentkoc
vincentkoc force-pushed the feat/plugin-resource-tokenjuice-20260923 branch from 4205705 to 9697f43 Compare September 23, 2026 12:36
@clawsweeper clawsweeper Bot added proof: sufficient Contributor real behavior proof is sufficient. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. and removed status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. labels Oct 8, 2026
@steipete

Copy link
Copy Markdown
Contributor

Thanks for the diagnostic work. Closing this draft as superseded by the reviewed owner repair in #502 (9bb9880), which includes phase observations plus native admission/cleanup controls and has passing required cross-platform CI. The remaining cold-start and test-timing uncertainty stays explicitly open in #305; this closure does not claim that a passing run explains every earlier failure.

@steipete steipete closed this Oct 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merge-risk: 🚨 automation 🚨 Merging this PR could break CI, automerge, proof capture, label sync, or automation. P2 Normal priority bug or improvement with limited blast radius. proof: sufficient Contributor real behavior proof is sufficient. rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants