Automate agentic workflow recompilation after gh-aw updates by IEvangelist · Pull Request #21022 · microsoft/aspire

David Pine (IEvangelist) · 2026-10-15T13:40:57Z

Description

Keep generated agentic workflows and compiler settings aligned after gh-aw action updates without requiring a maintainer to recompile them locally.

Use pull_request: closed for merged upstream Dependabot PRs targeting the default branch, including grouped GitHub Actions updates. No pull_request_target trigger or dangerous-trigger suppression is used.
Support manual dispatch to recompile with the newest gh-aw release already pinned in the default branch's workflow YAML—not the latest upstream release.
Verify action release comments against upstream commit SHAs, synchronize Copilot and validation compiler pins, install the exact release, compile all workflows without running gh aw upgrade, and reapply the auto-sec publication guard to the regenerated lock file, as validation does.
Use Aspirey and the existing local PR action to create or update follow-up PRs. Skip empty diffs and superseded updates, reject unexpected changed paths, and prevent publication after compilation, publication-guard, or lint failures.
For same-repo Dependabot PRs whose setup-cli pin is ahead of its version input, validate-agentic-workflows.yml still compiles and lints with the pinned compiler, which reverts the new gh-aw action pins. The strict generated-file check is replaced by one that tolerates differences in gh-aw action pin lines only; any other drift fails. The recompile PR created after the merge aligns the compiler and is validated strictly.

Fixes # (issue)

Limitation

Dependabot groups all GitHub Actions updates into one PR. A grouped PR that also bumps other actions pinned in generated workflows (for example actions/upload-artifact in #21048) still fails validation, because recompilation reverts those pins and only gh-aw pin lines are tolerated. A separate Dependabot group for github/gh-aw-actions* would let gh-aw-only bumps pass; it is not part of this PR.

Security considerations

Both execution paths check out only the trusted default branch; the Dependabot path requires a closed, merged upstream Dependabot PR, and manual dispatch is restricted to the default branch. GITHUB_TOKEN is explicitly limited to Contents and Pull requests read permissions. Write credentials are minted only after successful compilation, linting, and changed-path validation. The tolerance in validate-agentic-workflows.yml is a hygiene control, not a security boundary: that workflow runs from the PR's own tree with read-only contents access, applies only to same-repo Dependabot PRs, and leaves every other PR strict.

Deployment prerequisites:

Configure ASPIRE_BOT_APP_ID and ASPIRE_BOT_PRIVATE_KEY as both Actions secrets (manual runs) and Dependabot secrets (Dependabot-triggered runs), using the same names. GitHub does not expose normal Actions secrets to Dependabot-triggered pull_request workflows, including manual reruns. Neither name was present in the repository-level Dependabot secrets listing; organization-level availability was not verified. See GitHub's guidance and zizmor's remediation.
The App must have Contents, Pull requests, and Workflows write permissions. Installation permissions could not be verified locally.
The implementation reuses existing action SHAs; newly introduced third-party SHAs in generated PRs still require repository/enterprise allow-list confirmation. The policy API returned HTTP 403 locally.

Test coverage changes

Merging main removes ValidateAgenticWorkflowsTests.cs, which #20612 deleted along with other tests that asserted on workflow YAML. Following .github/instructions/infrastructure-tests.instructions.md, this PR drops the two YAML-asserting tests it had added to that file:

Removed test	Regression it targeted	Remaining coverage	Gap
`RecompilationUsesTrustedCheckoutAndDefersWriteCredentials`	`recompile-agentic-workflows.yml` checking out untrusted content, or minting write credentials before compile and lint	None	Not test-covered; beyond actionlint and the zizmor CI check, nothing asserts these properties
`PendingDependabotCompilerUpdatesSkipOnlyRegenerationAndDriftChecks`	The Dependabot exemption skipping more than the generated-file comparison	`checkGeneratedDrift` and `compilerUpdatePending` behavior tests	Workflow step gating (`if:` conditions) is not test-covered

RecompilationAutomationHandlesDependabotUpdates moved to RecompileAgenticWorkflowsScriptTests.cs. It runs the Node suite through NodeCommand and keeps the AgenticWorkflow trait, so the validation workflow's contract step still runs it. Eight new Node tests call checkGeneratedDrift and the generated-drift command against real temporary git repositories: tolerated gh-aw pin lines, a grouped update whose other action pins are rejected, changed, added, and removed lines, untracked, deleted, and action-cache changes, mode and binary changes, and output capping.

The publication-guard.js step in recompile-agentic-workflows.yml has no workflow-level test either. AutoSecWorkflowTests cover the guard's behavior, and strict validation of the follow-up PR reapplies the guard, so a missing step would surface there as generated-file drift.

Validation

.\restore.cmd — passed; installed the 11.0.100-rc.2.26504.105 SDK that global.json pins after the merge.
node --test tests\Infrastructure.Tests\WorkflowScripts\recompile-agentic-workflows.test.js — passed, 33 tests (25 existing, 8 new). Loosening the pin classifier to accept any uses: line failed exactly the three tests covering the grouped-update, other-differences, and command cases; the change was reverted.
.\.dotnet\dotnet.exe test --project tests\Infrastructure.Tests\Infrastructure.Tests.csproj --no-launch-profile -- --filter-class '*.RecompileAgenticWorkflowsScriptTests' --filter-not-trait 'quarantined=true' --filter-not-trait 'outerloop=true' — passed, 1 test.
The same command with --filter-trait "Category=AgenticWorkflow" in place of the class filter, which is the validation workflow's contract selection — 754 passed, 2 failed: ProcessRunnerTests.CapturesStreamsExitCodeAndInvocationContext and ProcessRunnerTests.AppliesEnvironmentOverridesOnTopOfInheritedEnvironment. Both fail only on Windows because bash resolves to the WSL launcher in System32, which does not forward the test's environment variables; running that launcher directly reproduces the failing output. These tests and ProcessRunner are identical to main.
actionlint.exe -shellcheck= -pyflakes= .github\workflows\validate-agentic-workflows.yml .github\workflows\recompile-agentic-workflows.yml using repository-pinned actionlint 1.7.12 — passed. zizmor.exe --offline --no-progress (1.30.1) on the same two files — no findings.
The added publication-guard.js step follows the same compile, guard, lint order as validate-agentic-workflows.yml. The job checks out the full default branch, and the guard locates auto-sec.lock.yml relative to its own directory, so the working directory does not matter. The step itself has not run: gh aw compile cannot be replayed locally on Windows.
Replayed the pending-update detector against the compiler files from the default branch before the gh-aw bump (not pending) and from Bump the github-actions group across 1 directory with 5 updates #21048 (pending).
Classified the generated-file changes in Bump the github-actions group across 1 directory with 5 updates #21048 (c9a9184a59..b71765ccbd): all 390 changed lines are uses: pin lines. The gh-aw setup and setup-cli pins are tolerated; the actions/download-artifact, actions/upload-artifact, and actions/setup-node pins are not, so that grouped PR still fails validation by design. Dependabot's own run on it (38888809369) failed the strict drift step on exactly nine generated files, all modified, with no untracked or deleted files.
git diff --check on the files this PR changes — passed.

The Dependabot-only steps in validate-agentic-workflows.yml have not run on a real runner; this PR's own run skips them and exercises the strict path. No full build or live Aspirey PR-creation run was performed.

Checklist

Is this feature complete?
- Yes. Ready to ship.
- No. Follow-up changes expected.
Are you including unit tests for the changes and scenario tests if relevant?
- Yes
- No
Did you add public API?
- Yes
  - If yes, did you have an API Review for it?
    - Yes
    - No
  - Did you add <remarks /> and <code /> elements on your triple slash comments?
    - Yes
    - No
- No
Does the change make any security assumptions or guarantees?
- Yes
  - If yes, have you done a threat model and had a security review?
    - Yes
    - No
- No

Handle merged Dependabot updates and manual dispatch with verified pinned compiler versions and Aspirey follow-up pull requests. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: cdfad3fd-5f51-4c95-ab2e-7e29d23b9268

github-actions · 2026-10-15T13:41:13Z

🚀 Dogfood this PR with:

⚠️ WARNING: Do not do this without first carefully reviewing the code of this PR to satisfy yourself it is safe.

curl -fsSL https://raw.githubusercontent.com/microsoft/aspire/main/eng/scripts/get-aspire-cli-pr.sh | bash -s -- 21022

Or

Run remotely in PowerShell:

iex "& { $(irm https://raw.githubusercontent.com/microsoft/aspire/main/eng/scripts/get-aspire-cli-pr.ps1) } 21022"

Scope the zizmor dangerous-triggers exception to the closed-PR trigger and document the merged Dependabot/default-branch execution boundary. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: cdfad3fd-5f51-4c95-ab2e-7e29d23b9268

Copilot

🟡 Changes recommended

Version capture blocks compilation, queued updates can be lost, and reused PRs can retain stale compiler metadata.

3 open findings

Capture gh-aw version output from stderr · New
Prevent pending workflow runs from being replaced · New
Update PR metadata when compiler changes · New

What changed in this PR

Automates Aspire’s agentic workflow recompilation after gh-aw updates, reducing manual compiler-pin synchronization and regeneration.

Changes:

Adds merged-Dependabot and manual triggers with trusted checkout and delayed write credentials.
Resolves pinned releases, synchronizes compiler settings, and validates generated changes before publication.
Adds behavior tests and operational documentation.

File	Description
tests/Infrastructure.Tests/WorkflowScripts/ValidateAgenticWorkflowsTests.cs	Integrates behavior tests and checks workflow safeguards.
tests/Infrastructure.Tests/WorkflowScripts/recompile-agentic-workflows.test.js	Tests release detection, synchronization, and publication guards.
.github/workflows/recompile-agentic-workflows.yml	Orchestrates compilation, validation, and follow-up PR creation.
.github/workflows/recompile-agentic-workflows.js	Implements release selection and changed-path validation.
.github/workflows/README.md	Documents automation, manual execution, and required permissions.

🧠 Review effort: Balanced

💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.

Remove the privileged trigger and its zizmor exception. Document the Dependabot secret requirements and guard closed merged PR events in the detector and tests. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: cdfad3fd-5f51-4c95-ab2e-7e29d23b9268

Copilot

🔵 Needs a closer look

Previously identified version-output and concurrency issues remain unresolved and can prevent required recompilation.

3 open findings

🧠 Review effort: Balanced

github-actions · 2026-10-15T14:23:17Z

Retrying the failed CI jobs for this pull request from the CI run attempt. The rerun is being tracked in the rerun attempt.

Capture version output from both streams, isolate concurrency by follow-up branch, and refresh existing PR metadata on manual recompilation. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: cdfad3fd-5f51-4c95-ab2e-7e29d23b9268

Copilot

🔵 Needs a closer look

Required Dependabot secrets, App write permissions, and live PR publication remain unverified.

0 open findings

3 resolved since last review

🧠 Review effort: Balanced

James Newton-King (JamesNK)

Review identified 1 correctness/workflow sequencing issue: the merged-only recompilation trigger cannot repair pin-only Dependabot PRs before existing generated-file validation rejects them. Validation: 19 new Node tests passed locally; all 8 workflows compiled with gh-aw v0.89.21 in an isolated copy and the validation recompilation produced no drift. Existing PR CI passed all 1,619 Infrastructure tests. No live token-minting or PR-publication test was run.

A Dependabot gh-aw-actions bump rewrites the pinned SHAs but not the version input that selects the compiler. Recompiling with the older compiler reverts the new setup pins, so the compile and drift checks cannot pass before merge. For same-repo Dependabot PRs whose setup-cli pin is ahead of its version input, skip regeneration and drift comparison. Lint and the contract tests still run, and the post-merge recompile PR aligns the compiler and is validated strictly. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: e6aa1dbb-e82e-4932-ab30-18094ee1ffc7

Copilot

🔵 Needs a closer look

Grouped-update correctness remains unresolved, and workflow-write publication requires human validation of credentials, permissions, and live PR creation.

1 open issue

Grouped updates bypass validation for non-gh-aw action changes · New

_{Review effort: Balanced}

…compile-agentic-workflows Accept upstream's deletion of ValidateAgenticWorkflowsTests.cs to resolve the modify/delete conflict. Upstream removed the YAML-restating infrastructure tests in #20612. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: e6aa1dbb-e82e-4932-ab30-18094ee1ffc7

The Dependabot exemption skipped regeneration and drift checks for the whole PR, so a grouped update that also bumped other actions in generated files (for example actions/checkout) could pass validation and then be reverted by the follow-up recompile. Keep compiling and validating with the pinned compiler. While a compiler update is pending, replace the strict drift check with one that accepts only gh-aw action pin lines and fails on any other difference. Cover it with real-git tests, and run the Node tests from a C# runner so they stay in the AgenticWorkflow contract category. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: e6aa1dbb-e82e-4932-ab30-18094ee1ffc7

Copilot

🟡 Changes recommended

Recompilation omits required auto-sec hardening, proposing removal of protections and causing follow-up PRs to fail strict validation.

1 open issue

Run publication guard after compilation to preserve workflow protections · New

1 resolved since last review

Grouped updates bypass validation for non-gh-aw action changes

_{Review effort: Balanced}

gh aw compile replaces auto-sec.lock.yml with raw compiler output, which removes its publication gates, private-stream suppression, and restricted artifact paths. Apply publication-guard.js after compiling and before linting, as validate-agentic-workflows.yml does, so recompilation PRs keep the hardened lock instead of proposing to strip it. Document the guard in the README's recompilation sequence. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: e6aa1dbb-e82e-4932-ab30-18094ee1ffc7

github-actions · 2026-10-20T21:03:38Z

Tests selector

1 / 99 PR test projects · 0 PR jobs, from 6 changed files.

Selected PR test projects (1 / 99)

Infrastructure.Tests

Selected PR jobs (0)

none

How these were chosen — grouped by what changed

📄 .github/workflows/README.md (changed)
→ 1 directly: Infrastructure.Tests

📄 .github/workflows/recompile-agentic-workflows.js (changed)
→ 1 directly: Infrastructure.Tests

📄 .github/workflows/recompile-agentic-workflows.yml (changed)
→ 1 directly: Infrastructure.Tests

📄 .github/workflows/validate-agentic-workflows.yml (changed)
→ 1 directly: Infrastructure.Tests

🧪 tests/Infrastructure.Tests/WorkflowScripts/RecompileAgenticWorkflowsScriptTests.cs (changed test)
→ 1 directly: Infrastructure.Tests

🧪 tests/Infrastructure.Tests/WorkflowScripts/recompile-agentic-workflows.test.js (changed test)
→ 1 directly: Infrastructure.Tests

Job reasons

none

Selection computed for commit 1e592f5.

Copilot

🔵 Needs a closer look

The live Dependabot publication path remains unvalidated, and required secret availability and App permissions are unconfirmed.

No open issues 🎉

1 resolved since last review

Run publication guard after compilation to preserve workflow protections

_{Review effort: Balanced}

Ankit Jain (radical) · 2026-10-22T17:11:27Z

[automated] Thanks for talking this through! As we discussed, the slash command sounds like a good way to handle regeneration on the original Dependabot PR before merging.

Ideally, it would update the compiler version, regenerate the workflows, and push the changes back to that PR, so everything goes through the existing validation together. I’d like us to keep that gate strict for Dependabot too, without relaxing the drift check or skipping cases.

That should save the manual regeneration work without leaving main temporarily out of sync between two merges. If a grouped dependency update can’t be reconciled, a clear failure explaining what needs attention seems better than making an exception in validation.

Sébastien Ros (sebastienros)

Two minor workflow issues are inline: one reliability, one efficiency. Both are fine as follow-ups, and I found no correctness bugs in the detection, drift-classification or publication code.

I'm leaving this as a comment instead of approving because radical's request is still unanswered. It asks to regenerate on the Dependabot PR and keep the drift check strict. It was posted after the latest commit, and this PR still relaxes that check, so that decision should come first.

Also worth weighing in that decision: .github/dependabot.yml puts all actions in one group, and the description says #21048 still fails validation by design. That PR has since closed unmerged, so the Dependabot path only helps once gh-aw has its own group or the design changes.

Checked:

The workflow reuses actions/checkout, actions/github-script and actions/create-github-app-token SHAs already used in this repo, and changes no pins, so this PR needs no allow-list update.
The 2>&1 version check matches gh-aw v0.89.17's gh aw version output.
A normal compile prunes stale gh-aw entries from actions-lock.json, so that file shouldn't drift from validation's --force-refresh-action-pins output. I read this in the gh-aw source and did not run it.

Not exercised: I reviewed the code only. As the description says, the Dependabot-only validation steps and the Aspirey publish path haven't run on a real runner, and the App's permissions and Dependabot secrets are unconfirmed.

Sébastien Ros (sebastienros) · 2026-11-04T03:24:43Z

+          # Compilation replaces auto-sec.lock.yml with raw output. Reapply its publication
+          # gates before linting, as validate-agentic-workflows.yml does.
+          node .github/workflows/auto-sec/publication-guard.js
+          gh aw lint --shellcheck


Nothing tells anyone when this job fails after a Dependabot merge, and that merge has already left main with setup-cli ahead of its version input. Until a follow-up PR lands, every non-Dependabot PR touching agentic inputs fails the strict drift check.

The failure is likely, not just possible. publication-guard.js targets v0.89.17 output and throws on anything it doesn't recognize, so a compiler bump that changes the output stops here with no PR. Missing Dependabot secrets would do the same at the token step. The only signal is a red run on a closed PR.

Please report the failure the way update-actionlint.yml does, with a failure() job calling report-pipeline-failure.js. A Dependabot-triggered run gets a read-only GITHUB_TOKEN, so filing the issue may need the App token. Fine as a follow-up.

Sébastien Ros (sebastienros) · 2026-11-04T03:24:43Z

+on:
+  workflow_dispatch:
+  pull_request:
+    types: [closed]


Without a paths filter, every PR closed in the repo starts a run of this workflow, only for the job-level if to skip it, which adds noise to the Actions history. detect only looks at .github/workflows/ YAML, so PRs that touch nothing there can't matter.

Suggested change

types: [closed]

types: [closed]

paths:

- '.github/workflows/**'

aspire-repo-bot Bot requested a balanced review from Copilot October 15, 2026 13:41

github-actions Bot added the area-engineering-systems infrastructure helix infra engineering repo stuff label Oct 15, 2026

github-advanced-security Bot found potential problems Oct 15, 2026

View reviewed changes

Comment thread .github/workflows/recompile-agentic-workflows.yml Fixed

Copilot started reviewing on behalf of David Pine (IEvangelist) October 15, 2026 13:42

View session

This comment has been minimized.

Sign in to view

Copilot AI reviewed Oct 15, 2026

View reviewed changes

Comment thread .github/workflows/recompile-agentic-workflows.yml Resolved

Comment thread .github/workflows/recompile-agentic-workflows.yml Resolved

Comment thread .github/workflows/recompile-agentic-workflows.yml Resolved

aspire-repo-bot Bot requested a balanced review from Copilot October 15, 2026 13:54

Copilot started reviewing on behalf of David Pine (IEvangelist) October 15, 2026 13:55

View session

This comment has been minimized.

Sign in to view

Copilot AI reviewed Oct 15, 2026

View reviewed changes

aspire-repo-bot Bot requested a balanced review from Copilot October 15, 2026 15:02

Copilot started reviewing on behalf of David Pine (IEvangelist) October 15, 2026 15:03

View session

This comment has been minimized.

Sign in to view

Copilot AI reviewed Oct 15, 2026

View reviewed changes

github-actions Bot mentioned this pull request Oct 15, 2026

[repo-pulse] Daily Report #16404

Open

James Newton-King (JamesNK) reviewed Oct 15, 2026

View reviewed changes

Comment thread .github/workflows/recompile-agentic-workflows.yml Resolved

aspire-repo-bot Bot requested a balanced review from Copilot October 19, 2026 13:28

Copilot started reviewing on behalf of David Pine (IEvangelist) October 19, 2026 13:29

View session

This comment has been minimized.

Sign in to view

Copilot AI reviewed Oct 19, 2026

View reviewed changes

Comment thread .github/workflows/validate-agentic-workflows.yml Resolved

David Pine (IEvangelist) and others added 2 commits October 20, 2026 15:41

This comment has been minimized.

Sign in to view

aspire-repo-bot Bot requested a balanced review from Copilot October 20, 2026 20:47

Copilot started reviewing on behalf of David Pine (IEvangelist) October 20, 2026 20:48

View session

Copilot AI reviewed Oct 20, 2026

View reviewed changes

Comment thread .github/workflows/recompile-agentic-workflows.yml Resolved

aspire-repo-bot Bot requested a balanced review from Copilot October 20, 2026 20:59

Copilot started reviewing on behalf of David Pine (IEvangelist) October 20, 2026 21:00

View session

Copilot AI reviewed Oct 20, 2026

View reviewed changes

Sébastien Ros (sebastienros) reviewed Nov 4, 2026

View reviewed changes

David Pine (IEvangelist) closed this Nov 11, 2026

Conversation

David Pine (IEvangelist) commented Oct 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Limitation

Security considerations

Test coverage changes

Validation

Checklist

Uh oh!

github-actions Bot commented Oct 15, 2026

Uh oh!

Uh oh!

This comment has been minimized.

This comment has been minimized.

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Uh oh!

Uh oh!

Uh oh!

Uh oh!

This comment has been minimized.

Copilot AI left a comment

Choose a reason for hiding this comment

🔵 Needs a closer look

Uh oh!

github-actions Bot commented Oct 15, 2026

Uh oh!

This comment has been minimized.

Copilot AI left a comment

Choose a reason for hiding this comment

🔵 Needs a closer look

Uh oh!

James Newton-King (JamesNK) left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

This comment has been minimized.

Copilot AI left a comment

Choose a reason for hiding this comment

🔵 Needs a closer look

Uh oh!

Uh oh!

This comment has been minimized.

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Uh oh!

Uh oh!

github-actions Bot commented Oct 20, 2026

Tests selector

Selected PR test projects (1 / 99)

Selected PR jobs (0)

Job reasons

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🔵 Needs a closer look

Uh oh!

Ankit Jain (radical) commented Oct 22, 2026

Uh oh!

Sébastien Ros (sebastienros) left a comment

Choose a reason for hiding this comment

Uh oh!

Sébastien Ros (sebastienros) Nov 4, 2026

Choose a reason for hiding this comment

Uh oh!

Sébastien Ros (sebastienros) Nov 4, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

David Pine (IEvangelist) commented Oct 15, 2026 •

edited

Loading