fix(workflows): make backlog grooming resilient to malformed model output

[Chris Montazer (rezatnoMsirhC)]

fix(workflows): make backlog grooming resilient to malformed model output

Description

This PR moves backlog-grooming transport and structural state into trusted scripts so one safely attributable malformed candidate cannot terminate an otherwise valid shard. Candidate identity, full partition coverage, provenance, and canonical digests remain fail-closed; malformed JSON is never heuristically repaired.

It also introduces candidate-addressed fixed-slot output, preserves non-advisory contract errors through checkpoints and reduction, and blocks both tracker and history publication unless the final aggregate has zero contract errors.

Related Issue(s)

Closes #2827
Closes #2866

Type of Change

Select all that apply:

Code & Documentation:

• Bug fix (non-breaking change fixing an issue)
• New feature (non-breaking change adding functionality)
• Breaking change (fix or feature causing existing functionality to change)
• Documentation update

Infrastructure & Configuration:

• GitHub Actions workflow
• Linting configuration (markdown, PowerShell, etc.)
• Security configuration
• DevContainer configuration
• Dependency update

AI Artifacts:

• Reviewed contribution with hve-builder and addressed all actionable findings
• Copilot instructions (.github/instructions/*.instructions.md)
• Copilot prompt (.github/prompts/*.prompt.md)
• Copilot agent (.github/agents/*.agent.md)
• Copilot skill (.github/skills/*/SKILL.md)
• Copilot hook (.github/hooks/*/*.json)
• Eval spec added/updated for changed AI artifacts (evals/)

Note for AI Artifact Contributors:

• Agents: Research, indexing/referencing other project (using standard VS Code GitHub Copilot/MCP tools), planning, and general implementation agents likely already exist. Review .github/agents/ before creating new ones.
• Skills: Must include both bash and PowerShell scripts. See Skills.
• Model Versions: Contributions MUST target models listed in the model catalog (scripts/linting/model-catalog.json) whose provider appears in providerAllowlist and whose status is ga or preview. Run npm run lint:models to validate references.
• See Agents Not Accepted and Model Version Requirements.

Other:

• Script/automation (.ps1, .sh, .py)
• Other (please describe):

Sample Prompts (for AI Artifact Contributions)

None

Testing

• 163 focused backlog-grooming Pester tests passed.
• 11 eval-text tests passed with 1 informational skip; the 1,144-file text scan reported 0 errors.
• gh aw compile backlog-groom --no-emit --validate --actionlint passed, and the generated lock was refreshed with gh aw v0.86.2.
• Eval schema, safety, and Vally lint passed; Markdown, frontmatter, YAML, plugin, AI-artifact, asset-doc, permission, runner, and dangerous-workflow checks passed.
• Complete feature-branch sweep: 148 candidates, 146 assessed, 2 deferred, 0 contract errors, and 0 normalizations.
• Read-only publisher validation authenticated the retained aggregate without making tracker or repository mutation reachable.
• Real publisher run revalidated the aggregate, created tracker #2897, and emitted immutable report provenance.

Checklist

Required Checks

• Documentation is updated (if applicable)
• Files follow existing naming conventions
• Changes are backwards compatible (if applicable)
• Tests added for new functionality (if applicable)

AI Artifact Contributions

• Used hve-builder review mode to review contribution
• Addressed all actionable findings from the hve-builder review
• Verified contribution follows common standards and type-specific requirements

Required Local Checks

The following local-safe validation commands must pass before merging:

• Local validation aggregate: npm run validate:local
• Documentation validation (if docs changed): npm run validate:docs
• Spell checking: npm run spell-check
• Link validation: npm run lint:md-links

Security Considerations

• This PR does not contain any sensitive or NDA information
• Any new dependencies have been reviewed for security issues
• Security-related scripts follow the principle of least privilege

Additional Notes

The report-history build and provenance validation passed in documentation run 34633656219. Final Pages deployment was rejected by the existing github-pages environment branch policy; no environment policy was changed. Temporary publication-test source changes were reverted before this PR.

[Codecov Comments Bot (codecov-commenter)]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 82.88%. Comparing base (9bf1a30) to head (87c7fa4).

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2901      +/-   ##
==========================================
- Coverage   83.01%   82.88%   -0.14%     
==========================================
  Files         186      172      -14     
  Lines       34939    34459     -480     
  Branches      231      201      -30     
==========================================
- Hits        29004    28560     -444     
+ Misses       5861     5828      -33     
+ Partials       74       71       -3     

Flag	Coverage Δ
docusaurus	?
pester	83.12% <ø> (-0.30%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.
see 16 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[github-actions]

Eval Execution

✅ Status: Passed — no merge-blocking failures (2 advisory assertion failure(s) present)

• Artifacts evaluated: 1
• Specs run: 1
• Assertions passed: 1
• Assertions failed (blocking): 0
• Assertions failed (advisory): 2
• Failed specs (merge-blocking): 0

Artifact	Kind	Status	Specs	Passed	Failed (blocking)	Failed (advisory)
github-backlog-grooming	instruction	⚠️ advisory-fail	1	1	0	2

Legend — ✅ clean · ⚠️ advisory failures only (non-blocking) · ⏭️ skipped · ❌ merge-blocking failure

Only Failed specs (merge-blocking) gates this PR. Advisory assertion failures are signal-quality checks captured during iteration; review them, but they do not block merge and may be acceptable.

[Bill Berry (WilliamBerryiii)]

Thank you for the substantial work on candidate isolation, trusted state derivation, v2 recovery, and publication gating. I completed a comprehensive review across functional, standards, accessibility, security, readiness, supply-chain provenance, and Responsible AI/community impact; six inline comments identify four blocking issues and two medium-risk gaps. The blocking items are fail-open handling of unbound result calls, a manual publisher path outside protected provenance, conflicting agent/workflow output contracts, and CRLF-sensitive tests (149/163 locally on Windows). Please also canonicalize supersession identities and define monitoring, escalation, and human oversight for community-facing outcomes; once addressed, rerun the focused tests and provenance checks.

[Bill Berry (WilliamBerryiii)]

F-001 (High): Fail closed on unbound result calls. This branch logs and drops a publication call whose identity cannot be bound to the planned shard. If valid calls cover the planned IDs, the aggregate still has zero contract errors and can publish, leaving the contradictory model output out of the durable record. Please throw before artifact creation for invalid or foreign identities, reserve candidate-local { issue, code } diagnostics for safely bound semantic failures, and add process tests proving no artifact is written for invalid, foreign, or conflicting identities.

[Bill Berry (WilliamBerryiii)]

F-002 (High): Bind the privileged publisher code to the provenance chain. These checks authenticate the aggregate and its upstream producer, but workflow_dispatch can execute this publisher from a caller-selected ref. That lets different workflow code receive issues: write or contents: write while processing a valid main-produced artifact. Please remove privileged manual dispatch, or use a no-write recovery wrapper that invokes a reusable publisher pinned to an approved full SHA behind a protected environment; attest and verify the exact subject digest, signer workflow revision, source ref/revision, event type, hosted runner, and builder identity before publication.

[Bill Berry (WilliamBerryiii)]

F-003 (High): Align the imported agent with the new scalar output contract. This still directs one JSON issues envelope, while the workflow requires one final scalar call per candidate and explicitly forbids serialized rows or arrays. Because both instructions enter the model context, the worker can follow the obsolete shape at the boundary this PR is hardening. Please replace this section with the candidate-addressed scalar contract and add a behavior check that rejects the old envelope.

[Bill Berry (WilliamBerryiii)]

F-004 (High): Make the workflow assertions line-ending agnostic. On a Windows checkout with core.autocrlf=true, the expected YAML is present but (?m)...$ leaves the preceding \r unmatched. The focused command therefore passed 149 of 163 tests, with all 14 failures in these changed text assertions. Please normalize Read-RepoFile output to LF or accept \r?$, then require the full focused suite to pass on Windows and Linux.

[Bill Berry (WilliamBerryiii)]

F-005 (Medium): Compare canonical lineage identities instead of prose. Byte inequality allows the same PR, issue, commit, or release to appear distinct through different casing, URLs, prefixes, or surrounding wording. That can preserve an unsupported Superseded outcome. Please bind evidence to structured canonical source identities, require disjoint original and replacement identity sets, and test equivalent references expressed in different forms.

[Bill Berry (WilliamBerryiii)]

F-006 (Medium): Define monitoring and escalation for community-facing outcomes. The production contract now documents the v2 validator path, but it does not define how maintainers monitor contract errors, normalizations, Deferred/Uncertain rates, dispositions, or maintainer overturns, nor who owns escalation and exceptional publication approval. Please document retained audit evidence, bounded measures and thresholds, escalation ownership, qualified-human gates, transition/replay handling, cleanup, and residual community risk.