Repository navigation
fix(ci): ci --config with a plugin config skips requested runtime capture - #125
Conversation
|
Codex review: needs maintainer review before merge. What this changesRoute plugin-root CI configs through the existing plugin check so requested runtime capture runs. Example: Run plugin-inspector ci --config plugin-inspector.config.json --no-openclaw --runtime --mock-sdk --allow-execute for a plugin registering one tool.
Review scores
ProductKind: Bug fix · Worth it: Yes · Fix scope: Complete Merge readiness✅ Ready for maintainer review This PR fixes a current CI routing bug, and no actionable defects remain; the updated Crabpot comparison resolves the previous review request. Priority: P2 Before mergeNone. FindingsNone. Agent review detailsHow this fits togetherThe CLI accepts plugin-root or fixture-suite configs, dispatches compatibility inspection and optional runtime capture, and writes compatibility reports and CI artifacts. flowchart TD
CLI[CI command] --> Config[Load and classify config]
Config --> Suite[Fixture suite inspection]
Config --> Plugin[Existing plugin check]
Plugin --> Consent[Execution consent check]
Consent --> Capture[Runtime capture]
Suite --> Reports[Compatibility and CI artifacts]
Plugin --> Reports
Capture --> Runtime[Runtime artifacts]
Technical reviewBest possible solution: Land the focused routing repair using the existing config discriminator and plugin-check owner. Do we have a high-confidence way to reproduce the issue? Current-main source deterministically bypasses runtime capture for explicit configs, corroborated by the contributor's before/after CLI trace; this read-only review ran no target code. Is this the best way to solve the issue? The patch repairs dispatch at its existing owner, preserves fixture-suite behavior and output roots, and avoids duplicating runtime capture. AGENTS.md: found and applied where relevant. Codex review notes: model internal, reasoning medium; reviewed against 8ec7d359fd78. Provenance checked
TestingProof path: shipped entry point. SecurityNone. EvidenceWhat I checked:
Likely related people:
Review metrics
LabelsLabel changes:
Label justifications:
Rating scale6/6 🦀 challenger crab · 5/6 🦞 diamond lobster · 4/6 🐚 platinum hermit · 3/6 🦐 gold shrimp · 2/6 🦪 silver shellfish · 1/6 🧂 unranked krab. Overall follows the weaker of proof and patch quality; ✨ marks media proof (a screenshot, video, or linked artifact) that directly shows the changed behavior. WorkflowClawSweeper edits this one comment on every review. Comment HistoryReview history (1 earlier review cycle)
Reviewed October 30, 2026, 3:26 AM ET / 07:26 UTC (Revision 2). |
|
@clawsweeper re-review |
|
🦞👀 Re-review progress:
|
|
Maintainer validation at
|
What Problem This Solves
plugin-inspector ci --config plugin-inspector.config.json --runtime --mock-sdk --allow-executereports PASS and exits 0 but never runs runtime capture, so a CI job that asked for runtime registration checks passes without them.cigets--config,runCiCompatibilityReport(src/cli.js) always takes the fixture-suite path (inspectCompatibilityFixtureSet) and never callsrunPluginCheck, even when the file is a plugin-root config like the oneinitwrites.--runtime,--mock-sdkand--allow-execute(all listed forciin--help) are dropped without a word.fixturesarray) take exactly the same path as before;ciwithout--config,check,inspectandreportare untouched; no report field or finding code changes. Without--runtimethe plugin-rootci --configoutput is identical to main (same report JSON apart fromgeneratedAt, same file list).cihonours the flags its help lists.User Impact
Plugin authors who point
ciat theirplugin-inspector.config.jsonand ask for--runtimenow get the runtime capture report (plugin-inspector-runtime-capture.json/.md), the same asciwithout--config, instead of a PASS that silently skipped it.Why This Change Was Made
runCiCompatibilityReportloads the--configfile as before; when it is a plugin-root config (isPluginRootConfig) it now runsrunPluginCheckwith that config path and the config's directory as the plugin root (the same root and report directory the old path used), so runtime capture and the config'scapturesettings apply. Fixture-suite configs keep the fixture path.--configbranch was written for fixture-suite configs;loadInspectorConfigalso accepts plugin-root configs (it normalises them into a one-fixture set), so they were sent down the fixture path that has no runtime capture.ci --configtests never passed--runtime. A new case intest/cli.test.jsrunsci --config <plugin-root config> --runtime --mock-sdk --allow-executeand expects one captured registration; with thesrc/cli.jschange reverted the whole suite goes from 835 passing to 834 passing / 1 failing (only the new case).npm test835/835 andnpm run release:contentspass on the branch. CHANGELOG line under## Unreleased.Evidence
ci --config <plugin-root config> --runtimeskipping runtime capture while exiting 0.npm ci --ignore-scripts; a scratch plugin root whose entry registers one tool, with its config written byplugin-inspector init;upstream/main8ec7d35 in a worktree versus this branch.node src/cli.js ci --config plugin-inspector.config.json --no-openclaw --runtime --mock-sdk --allow-executefrom the plugin root, then a look atreports/.node src/cli.js ci --config …with--runtimeon main writes no runtime capture at all; on this branch it writesplugin-inspector-runtime-capture.json/.mdwith the one registration, matching whatcialready does without--config(the control).20689eawith every plugin submodule checked out,CRABPOT_PLUGIN_INSPECTOR_CLI=source CRABPOT_PLUGIN_INSPECTOR_DIR=<checkout> npm run plugin-inspector:smoke, run once against this branch and once againstupstream/main8ec7d35. The smoke drivesreport --config <fixture suite>, which this change does not touch, and both runs print the same result line for line (Status: FAIL,Fixtures: 59,Breakages: 35, the same top findings; only the Node PID in an experimental-feature warning differs). The 35missing-expected-seambreakages are the same on main in this local setup, so they are not caused by this PR.--resources,--retirements, packed package).