RunTests action

[spetersenms]

Summary

Adds the preview useSeparateTestAction setting, defaulting to false, which moves normal test execution from RunPipeline into a new RunTests action using al runtests.

Flow

1. RunPipeline builds and publishes the apps, creates the container, and keeps it alive.
2. It exports the container name, generated credentials, and a runtime signal.
3. The workflow conditionally invokes RunTests.
4. RunTests selects the test apps, executes them with AlTool, and produces JUnit results for the existing analysis step.
5. The existing cleanup step removes the container.

Notes

• Only normal tests are moved. BCPT, page scripting, and multi-country tests remain in RunPipeline.
• runTestsInAllInstalledTestApps uses the existing installTestAppsJson handoff rather than discovering every installed server app.
• The optional testType setting supports UnitTest, IntegrationTest, and Uncategorized.
• A custom RunTestsInBcContainer override can still replace the built-in AlTool runner.
• BCH is still used for container creation, app installation and metadata, server and company discovery, test enumeration, and event-log collection. AlTool owns the actual test execution.

✅ Checklist

• Add tests (E2E, unit tests)
• Update RELEASENOTES.md
• Update documentation (e.g. for new settings or scenarios)
• Add telemetry

[Copilot]

Pull request overview

Adds an opt-in RunTests action that executes normal tests separately against the RunPipeline container.

Changes:

• Adds separate test-action workflow wiring and settings.
• Implements AlTool execution, JUnit output, overrides, and diagnostics.
• Adds documentation and comprehensive Pester coverage.

Reviewed changes

Copilot reviewed 18 out of 18 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
Tests/RunTests.Test.ps1	Tests test selection and execution.
Tests/RunTests.Action.Test.ps1	Tests action wiring and credentials.
Tests/AnalyzeTests.Test.ps1	Tests generated JUnit analysis.
Tests/AlToolTestRunner.Test.ps1	Tests AlTool runner behavior.
Templates/Per Tenant Extension/.github/workflows/_BuildALGoProject.yaml	Adds RunTests workflow step.
Templates/AppSource App/.github/workflows/_BuildALGoProject.yaml	Adds RunTests workflow step.
Scenarios/settings.md	Documents the new setting.
RELEASENOTES.md	Announces separate test execution.
Actions/RunTests/RunTests.psm1	Selects and runs test apps.
Actions/RunTests/RunTests.ps1	Implements the action entry point.
Actions/RunTests/README.md	Documents the action.
Actions/RunTests/AlToolTestRunner.psm1	Implements AlTool and JUnit integration.
Actions/RunTests/action.yaml	Defines the composite action.
Actions/RunPipeline/RunPipeline.ps1	Keeps eligible containers alive.
Actions/RunPipeline/README.md	Documents handoff variables.
Actions/AnalyzeTests/TestResultAnalyzer.ps1	Distinguishes failures from skipped tests.
Actions/.Modules/settings.schema.json	Adds setting schema metadata.
Actions/.Modules/ReadSettings.psm1	Adds the default setting.

Suppressed comments (2)

Actions/RunPipeline/RunPipeline.ps1:559

• -keepContainer disables Run-AlPipeline's normal teardown, but the new RunTests action and both workflows never invoke Remove-BcContainer. Every opt-in build therefore leaves the BC container running; if RunPipeline fails, RunTests is skipped and cannot perform cleanup either. Add an always-run cleanup path after event-log capture for both build and test failure paths, otherwise persistent self-hosted runners will accumulate containers.

        -keepContainer:$runTestsInSeparateAction `

Actions/RunTests/RunTests.ps1:88

• Convert settings recursively here. Without -recurse, nested settings remain PSCustomObjects rather than the hashtables expected by the settings-processing helpers, making the separate action behave differently for nested settings.

$settings = $env:Settings | ConvertFrom-Json | ConvertTo-HashTable

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Maria Zhelezova (mazhelez)]

📦 Release Notes Update Required

AL-Go v9.2 has been released, and your changes to RELEASENOTES.md appear to be under that version's section.

Action needed: Please move your release notes entry to above the ## v9.2 heading so it will be included in the next release.

Example

## Changes to be included in the next release

- Your change here ✅

## v9.2

- Already released changes

Thank you for contributing to AL-Go! 🙏