Skip to content

refactor(skills): correct RPI and HVE Builder subagents to be optional and advisory and remove rpi-quick and hve-builder-tester skills - #2902

Open
Allen Greaves (agreaves-ms) wants to merge 4 commits into
mainfrom
feat/rpi-subagent-changes
Open

Allen Greaves (agreaves-ms) wants to merge 4 commits into
mainfrom
feat/rpi-subagent-changes

Conversation

@agreaves-ms

Copy link
Copy Markdown
Collaborator

Pull Request

Description

Simplifies how the RPI and HVE Builder workflows use subagents, and removes lifecycle machinery that had become a source of confusion.

RPI subagents become optional and advisory. No RPI phase requires a subagent. RPI Researcher and the renamed RPI Reviewer (formerly RPI Review Builder) are read-only helpers the phase parent may assign a bounded, context-heavy portion of work to. They return source pointers or candidate findings as suggestions; the parent verifies at the cited evidence, assigns every C#, W#, or RV-xxx ID, and owns all artifacts, decisions, and routes. Both run on GPT-5.6 Luna (copilot). The RPI Planner subagent is removed; rpi-plan instead watches for skills or subagents whose descriptions say they are used during planning and follows those descriptions.

rpi-implement is refocused on following the plan, updating it as new information comes to light, checking off completed work, and keeping a condensed behavioral changes log. It no longer mentions helpers or prescribes how the model should implement code.

rpi-research defaults to balanced posture (was deepest) and exposes posture= as a caller argument.

rpi-quick is removed along with all references. RPI Agent and direct phase skills are the entry surfaces.

hve-builder moves to a parent-owned review pass. The hve-builder-tester skill, HVE Artifact Tester subagent, and all freeze, sandbox, fidelity, change-classification, and behavior-gate instructions are removed. The main agent reviews its candidate against the requirements catalog and review rubric, optionally dispatching the new read-only HVE Builder Reviewer subagent (GPT-5.6 Luna) for a fresh-context pass. The parent decides what passes: it may accept or reject any reviewer finding on its own reasoning, including a finding that calls an instruction confusing when the parent's reading shows it is suitable for its purpose, and records the disposition in the review evidence.

Caller cleanup. Eighteen non-RPI agents and instructions stop restating what the rpi-research skill already owns. hve-builder gains guidance for authoring RPI phase extensions where the artifact description is the whole contract.

Related Issue(s)

N/A

Type of Change

Select all that apply:

Code & Documentation:

  • Bug fix (non-breaking change fixing an issue)
  • New feature (non-breaking change adding functionality)
  • Breaking change (fix or feature causing existing functionality to change)
  • Documentation update

Infrastructure & Configuration:

  • GitHub Actions workflow
  • Linting configuration (markdown, PowerShell, etc.)
  • Security configuration
  • DevContainer configuration
  • Dependency update

AI Artifacts:

  • Reviewed contribution with hve-builder and addressed all actionable findings
  • Copilot instructions (.github/instructions/*.instructions.md)
  • Copilot prompt (.github/prompts/*.prompt.md)
  • Copilot agent (.github/agents/*.agent.md)
  • Copilot skill (.github/skills/*/SKILL.md)
  • Copilot hook (.github/hooks/*/*.json)
  • Eval spec added/updated for changed AI artifacts (evals/)

Other:

  • Script/automation (.ps1, .sh, .py)
  • Other (please describe):

Sample Prompts (for AI Artifact Contributions)

User Request:

Improve the code-review skill so it records skipped validation checks with a reason, and consolidate the duplicated stop rules.

Execution Flow:

  1. hve-builder resolves the target, infers create,improve,refactor, captures the current contract, and authors the change directly.
  2. Runs local validation (validate:skills, lint:frontmatter) and closes any mechanical findings.
  3. Runs the review pass. Because the change alters a stage gate, it dispatches HVE Builder Reviewer with the target, requirements, catalog, rubric, and read-only boundary.
  4. The reviewer returns, for example, one High required finding (a stop rule still contradicts the new skip-with-reason behavior in a reference file) and one Low advisory finding (a sentence it found unclear).
  5. The parent reads both cited locations, confirms the High finding and fixes it, rejects the Low finding with the reason that the sentence is consistent with the skill's stated purpose and repository conventions, reruns affected checks, and closes the High finding ID with a targeted re-review.
  6. Records the review verdict, dispositions, and rejection reason under .copilot-tracking/hve-builder/{{YYYY-MM-DD}}/ and returns Pass.

For RPI, rpi-review may hand RPI Reviewer one assignment in plain words ("compare the P02 and P03 Requirements: blocks against the changes record and report any requirement without matching evidence") with the two file paths, then verifies each returned candidate before recording an RV-xxx.

Output Artifacts:

.copilot-tracking/hve-builder/2026-09-11/code-review-author-1.md
.copilot-tracking/hve-builder/2026-09-11/code-review-review-1.md

Review evidence excerpt:

## Review Pass
Reviewer: HVE Builder Reviewer (dispatched; fresh context)
Suggested verdict: Revise (advisory)

| ID    | Dimension         | Severity | Disposition | Parent decision                              |
|-------|-------------------|----------|-------------|----------------------------------------------|
| R-001 | Workflow contract | High     | Required    | Accepted; fixed in references/review-flow.md |
| R-002 | Outcome/structure | Low      | Advisory    | Rejected; wording matches stated purpose     |

Recorded verdict: Pass (revision after R-001 closure)

Success Indicators:

  • No RPI phase or hve-builder stage blocks on a subagent being available.
  • Subagent returns contain no C#, W#, or RV-xxx IDs and no file writes; the parent's artifact carries all IDs and decisions.
  • hve-builder review evidence names the reviewed revision and records each reviewer finding as accepted or rejected with a reason.
  • rpi-research opens at balanced posture unless posture= or a recorded brief-based reason changes it.
  • plugin.json, extension/, docs/reference/, and evals/agent-behavior/ regenerate with no drift.

Testing

  • npm run validate:skills, npm run lint:frontmatter, npm run lint:md, npm run lint:md-links, npm run lint:tables, npm run plugin:validate, npm run docs:generate:check all pass.
  • scripts/evals/Build-AgentBehaviorSpec.ps1 -WhatIf and Build-AgentInventory.ps1 -WhatIf report no drift (62 agents, 30 subagents).
  • git diff --check clean.
  • Regenerated: plugin.json (59 agents, 76 skills), docs/reference/ catalogs and pages, extension/package.json, extension/README.md, evals/agent-behavior/AGENTS.yml and eval.yaml.
  • evals/behavior-conformance/skill-behavior.eval.yaml: removed rpi-quick and hve-builder-tester stimuli, rewrote the hve-builder review-routing and convergence stimuli; total 228.
  • New stimulus partials for hve-builder-review and rpi-reviewer; rpi-planner and hve-artifact-tester partials and the orphaned tester fixture removed.
  • Repository sweeps for rpi-quick, rpi-planner, hve-builder-tester, hve-artifact-tester, RPI Review Builder, .copilot-tracking/sandbox, and freeze/behavior-gate vocabulary return only CHANGELOG, logs, tracking, synthetic test fixtures, and one historical ADR.
  • Self-reviewed each rewritten skill and subagent end to end; the behavior-test gate was waived by the maintainer for this change set.

Checklist

Required Checks

  • Documentation is updated (if applicable)
  • Files follow existing naming conventions
  • Changes are backwards compatible (if applicable)
  • Tests added for new functionality (if applicable)

AI Artifact Contributions

  • Used hve-builder review mode to review contribution
  • Addressed all actionable findings from the hve-builder review
  • Verified contribution follows common standards and type-specific requirements

Required Local Checks

The following local-safe validation commands must pass before merging:

  • Local validation aggregate: npm run validate:local
  • Documentation validation (if docs changed): npm run validate:docs
  • Spell checking: npm run spell-check
  • Link validation: npm run lint:md-links

Security Considerations

  • This PR does not contain any sensitive or NDA information
  • Any new dependencies have been reviewed for security issues
  • Security-related scripts follow the principle of least privilege

Additional Notes

  • Breaking: rpi-quick and hve-builder-tester skills, and the RPI Planner and HVE Artifact Tester subagents, are removed. RPI Review Builder is renamed RPI Reviewer. Callers referencing the old names need updating.
  • .copilot-tracking/sandbox/ is no longer a tracking root; its applyTo entry was dropped from copilot-tracking.instructions.md.
  • Agent tools: frontmatter on RPI Researcher was intentionally left unchanged per the hve-builder tool-configuration boundary, even though the body is now read-only.
  • docs/planning/adrs/0010-*.md still names hve-builder-tester as a historical run record and was left as-is.
  • docs/reference/README.md skill count moved 78 to 79 during regeneration; this was pre-existing generator drift picked up by the run.
  • npm run validate:local, validate:docs, and spell-check were not run for this body; the individual lint and drift checks listed under Testing were.

…and rpi-quick, default research to balanced

- remove all required subagent dispatch from rpi-research/plan/implement/review; helpers are optional and return suggestions only
- delete RPI Planner subagent and rpi-quick skill; set RPI Researcher and Review Builder to GPT-5.6 Luna
- refocus rpi-implement on following/updating the plan, checking off work, and a condensed changes log
- add posture= arg to rpi-research (balanced default); add hve-builder guidance for RPI phase extensions
- strip skill-restating text from 18 callers; update docs, evals, plugin.json, and extension projections

🧹 - Generated by Copilot
… pass, rename RPI Review Builder to RPI Reviewer

- remove hve-builder-tester skill, HVE Artifact Tester subagent, and all freeze/sandbox/behavior-gate instructions
- add HVE Builder Reviewer subagent (GPT-5.6 Luna) as an optional fresh-context reviewer; parent accepts or rejects findings and owns the verdict
- rename rpi-review-builder to rpi-reviewer as an assignment-based helper needing no RV/Pxx inputs; rpi-review verifies its findings
- update aliases, instructions, docs, evals (228 stimuli), plugin.json, and extension projections

🧹 - Generated by Copilot
@codecov-commenter

Codecov Comments Bot (codecov-commenter) commented Sep 11, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 83.00%. Comparing base (9569d84) to head (ef7f8c5).

Additional details and impacted files

Impacted file tree graph

@@           Coverage Diff           @@
##             main    #2902   +/-   ##
=======================================
  Coverage   83.00%   83.00%           
=======================================
  Files         185      185           
  Lines       34924    34924           
  Branches      228      228           
=======================================
  Hits        28989    28989           
  Misses       5861     5861           
  Partials       74       74           
Flag Coverage Δ
docusaurus 90.60% <ø> (ø)
pester 83.41% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.
see 2 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@github-actions

github-actions Bot commented Sep 11, 2026

Copy link
Copy Markdown
Contributor

Eval Execution

Status: Failed — 1 spec(s) block merge

  • Artifacts evaluated: 37
  • Specs run: 37
  • Assertions passed: 285
  • Assertions failed (blocking): 0
  • Assertions failed (advisory): 73
  • Failed specs (merge-blocking): 1
Artifact Kind Status Specs Passed Failed (blocking) Failed (advisory)
accessibility-planner agent ✅ pass 1 5 0 0
code-review agent ⏭️ skipped 1 68 0 12
code-review-walkback agent ✅ pass 1 5 0 0
pptx agent ✅ pass 1 5 0 0
documentation agent ⏭️ skipped 1 9 0 1
rpi-agent agent ⏭️ skipped 1 10 0 5
hve-builder-review agent ✅ pass 1 5 0 0
rpi-researcher agent ✅ pass 1 5 0 0
rpi-reviewer agent ✅ pass 1 5 0 0
privacy-planner agent ✅ pass 1 5 0 0
privacy-reviewer agent ⚠️ advisory-fail 1 4 0 1
brd-builder agent ⏭️ skipped 1 15 0 0
network-isa95-planner agent ✅ pass 1 5 0 0
prd-builder agent ⏭️ skipped 1 15 0 0
rai-planner agent ✅ pass 1 10 0 0
security-planner agent ⚠️ advisory-fail 1 5 0 30
sssc-planner agent ✅ pass 1 10 0 0
accessibility-identity instruction ⚠️ advisory-fail 1 0 0 3
copilot-tracking instruction ✅ pass 1 3 0 0
hve-builder instruction ✅ pass 1 3 0 0
adr-standards instruction ✅ pass 1 3 0 0
rai-identity instruction ⚠️ advisory-fail 1 2 0 4
identity instruction ✅ pass 1 3 0 0
sssc-planner instruction ✅ pass 1 3 0 0
standards-mapping instruction ✅ pass 1 3 0 0
hve-artifact-authoring skill ✅ pass 1 3 0 0
hve-builder skill ✅ pass 1 21 0 0
prompt-analyze skill ✅ pass 1 9 0 0
prompt-builder skill ⚠️ advisory-fail 1 7 0 2
prompt-refactor skill ⚠️ advisory-fail 1 5 0 4
rpi-challenger skill ⚠️ advisory-fail 1 0 0 3
rpi-implement skill ⚠️ advisory-fail 1 5 0 4
rpi-plan-critique skill ✅ pass 1 3 0 0
rpi-plan skill ✅ pass 1 9 0 0
rpi-research skill ⚠️ advisory-fail 1 8 0 1
rpi-review skill ⚠️ advisory-fail 1 6 0 3
rpi-walkthrough skill ✅ pass 1 3 0 0

Legend — ✅ clean · ⚠️ advisory failures only (non-blocking) · ⏭️ skipped · ❌ merge-blocking failure

Only Failed specs (merge-blocking) gates this PR. Advisory assertion failures are signal-quality checks captured during iteration; review them, but they do not block merge and may be acceptable.

🧪 - Generated by Copilot

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants