Skip to content
Draft
Show file tree
Hide file tree
Changes from 9 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 10 additions & 3 deletions .github/instructions/ado-pipeline.instructions.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,11 +35,18 @@ Every ADO pipeline in this repo is split into **two YAML files**:

`ob_*` job-scope variables and `LinuxContainerImage` still appear here (ADO requires them at job scope), but they are set from `${{ parameters.* }}`, so the raw author never has to know which OneBranch convention they satisfy.

File-pairing convention: a wrapper at `.github/workflows/ado/<name>.yml` pairs with a raw stages template at `.github/workflows/ado/templates/<stem>-stages.yml`. **Multiple wrappers MAY share a single raw stages template** — that is in fact a primary motivation for the split: define several ADO pipelines (e.g. a DEV NonOfficial wrapper and a PROD Official wrapper, or per-environment variants) that all run the same stages/jobs/steps but differ in OneBranch variant, `featureFlags`, service connection, variable group, container image, etc. When wrappers share a raw template, name them so the relationship is obvious (e.g. `sources-upload-dev.yml` + `sources-upload-prod.yml` both pointing at `templates/sources-upload-stages.yml`). The variant choice cannot be hoisted into a shared sub-template because ADO requires `extends:` at the root of the entry pipeline — that is exactly why each wrapper exists.
File-pairing convention: a wrapper at `.github/workflows/ado/<name>.yml` pairs with a raw stages template at `.github/workflows/ado/templates/<stem>-stages.yml`. **Multiple wrappers MAY share a single raw stages template** — that is in fact a primary motivation for the split: define several ADO pipelines (e.g. a DEV NonOfficial wrapper and a PROD Official wrapper, or per-environment variants) that all run the same stages/jobs/steps but differ in OneBranch variant, `featureFlags`, service connection, variable group, container image, etc. When wrappers share a raw template, name them so the relationship is obvious (e.g. `package-build-dev.yml` + `package-build-prod.yml` both pointing at `templates/package-build-stages.yml`). The variant choice cannot be hoisted into a shared sub-template because ADO requires `extends:` at the root of the entry pipeline — that is exactly why each wrapper exists.

See [.github/workflows/ado/sources-upload.yml](.github/workflows/ado/sources-upload.yml) and [.github/workflows/ado/templates/sources-upload-stages.yml](.github/workflows/ado/templates/sources-upload-stages.yml) for the canonical example.
See [.github/workflows/ado/package-build.yml](.github/workflows/ado/package-build.yml) and [.github/workflows/ado/templates/package-build-stages.yml](.github/workflows/ado/templates/package-build-stages.yml) for the canonical example.

Shared **step sub-templates** live under `.github/workflows/ado/templates/steps/<name>.yml` and are spliced into a job's `steps:` via `- template: steps/<name>.yml` (path relative to the including stages template). Use these to share step sequences across stages templates that differ only in a trailing pipeline-specific step. Splicing as **steps** (not a separate job/stage) keeps job-scoped pipeline variables and on-disk files flowing to the steps that follow — a separate job would force output variables + artifact upload/download. The including job must define any job-scope variables the shared steps reference (e.g. `ob_outputDirectory`). See [.github/workflows/ado/templates/steps/common-steps.yml](.github/workflows/ado/templates/steps/common-steps.yml), shared by the `package-build` and `sources-upload` pipelines.
Shared **step sub-templates** live under `.github/workflows/ado/templates/steps/<name>.yml` and are spliced into a job's `steps:` via `- template: steps/<name>.yml` (path relative to the including stages template). Use these to share step sequences across stages templates that differ only in a trailing pipeline-specific step. Splicing as **steps** (not a separate job/stage) keeps job-scoped pipeline variables and on-disk files flowing to the steps that follow — a separate job would force output variables + artifact upload/download. The including job must define any job-scope variables the shared steps reference (e.g. `ob_outputDirectory`). See the granular templates under [.github/workflows/ado/templates/steps/](.github/workflows/ado/templates/steps/) (e.g. `ensure-full-history.yml`, `install-deps.yml`, `prepare-change-set.yml`), shared by the `package-build` (via `common-steps.yml`) and `pr-check-ct` pipelines.

**Parameterize the names of pipeline variables a template sets.** If a step template sets a job variable (`##vso[task.setvariable variable=<name>...]`), expose `<name>` through a parameter (with the conventional name as the default) so a caller composing several templates can rename it to avoid collisions. Use a consistent suffix scheme:

- **Same-job variables** (default `##vso[task.setvariable]`, no `isoutput`): parameter `<var_name>Var` (e.g. `sourceCommitVar`, `changedComponentsFileVar`).
- **Output variables** (`isoutput=true`, visible in other jobs/stages): parameter `<var_name>OutputVar`, AND the setting task's `name:` must also be parameterized (a cross-job output variable is referenced as `<taskName>.<var>`, so the task name is part of the contract).
Comment thread
PawelWMS marked this conversation as resolved.
Outdated

Reading another template's variable follows the same shape: take a `<var_name>Var` parameter and reference it as `$(${{ parameters.<var_name>Var }})` in `env:` (e.g. `prepare-change-set.yml`'s `fromCommitVar`/`toCommitVar`).

## OneBranch templates (MANDATORY — wrapper only)

Expand Down
7 changes: 3 additions & 4 deletions .github/workflows/ado/package-build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,15 +7,14 @@
# .github/workflows/ado/templates/package-build-stages.yml
#
# Authenticates via Workload Identity Federation (OIDC) and calls the Control
# Tower APIs to run the v1 post-merge delta build (stopgap until source upload
# is reworked):
# Tower APIs to run the v1 post-merge delta build:
# 1. Resolve the (target, source) commit range for this push from the
# previous CI build (ADO Builds API).
# 2. Submit official package builds for the components that changed across
# that range, via Workload Identity Federation (OIDC) to Control Tower.
#
# The setup + change-detection + validation steps are shared with the
# source-upload pipeline via templates/steps/common-steps.yml.
# The setup + change-detection + validation steps are composed from the shared
# templates/steps/* step templates via templates/steps/common-steps.yml.
#
# Helper scripts live under:
# - scripts/ci/control-tower/ - (Control Tower-specific)
Expand Down
Original file line number Diff line number Diff line change
@@ -1,16 +1,17 @@
# Microsoft Corporation
#
# Wrapper pipeline — passed to ADO as the entry point for the PR package-build
# check. It submits a *scratch* Control Tower build of the components a pull
# request changes, WAITS for that build to finish, and fails the check if the
# build fails (or is rejected). The build runs in Control Tower's own sandbox —
# NO PR-controlled code is built on the CI agent (only read-only change
# detection runs there).
# Wrapper pipeline — passed to ADO as the entry point for the PR Control Tower
# check. It calls Control Tower 'prcheck' (which uploads the changed
# components' missing lookaside sources) and then submits a *scratch* Control
# Tower build of those components, WAITS for that build to finish, and fails
# the check if the build fails (or is rejected). The build runs in Control
# Tower's own sandbox — NO PR-controlled code is built on the CI agent (only
# read-only change detection runs there).
#
# This file owns all OneBranch-specific wiring (governed templates repo,
# NonOfficial variant, featureFlags) and delegates the actual stages/jobs/steps
# to the raw stages template at:
# .github/workflows/ado/templates/pr-package-build-stages.yml
# .github/workflows/ado/templates/pr-check-ct-stages.yml
#
# WHY SCRATCH + REVIEWER-GATED: building unmerged PR code is only safe because
# (a) it is a *scratch* build (throwaway, never persisted to a production
Expand Down Expand Up @@ -56,6 +57,12 @@ trigger: none

pr: none

# Source the Control Tower API audience + base URL from the variable group here,
# in the wrapper (the env-specific entry point), and pass them to the stages
# template as parameters. The stages template stays variable-group-agnostic.
Comment thread
PawelWMS marked this conversation as resolved.
Outdated
variables:
- group: ControlTower-PRCheck

resources:
repositories:
- repository: templates
Expand Down Expand Up @@ -84,14 +91,15 @@ extends:
enabled: false

stages:
- template: /.github/workflows/ado/templates/pr-package-build-stages.yml@self
- template: /.github/workflows/ado/templates/pr-check-ct-stages.yml@self
parameters:
outputDirectory: $(Build.ArtifactStagingDirectory)/output
artifactBaseName: prpackagebuild
artifactBaseName: prcheckct
containerImage: mcr.microsoft.com/onebranch/azurelinux/build:3.0
poolType: linux
serviceConnection: CT-Endpoints-Access-ServiceConnection-DEV
variableGroup: ControlTower-PRCheck
apiAudience: $(ApiAudience)
apiBaseAFDUrl: $(ApiBaseAFDUrl)
# Control Tower package target for the 4.0 branch.
packageTarget: azl4
# This check WAITS for the Control Tower build to finish (pass/fail),
Expand Down
82 changes: 0 additions & 82 deletions .github/workflows/ado/sources-upload.yml

This file was deleted.

6 changes: 3 additions & 3 deletions .github/workflows/ado/templates/package-build-stages.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,9 @@
# OneBranch-coupled knobs as parameters. The wrapper at
# .github/workflows/ado/package-build.yml supplies concrete values.
#
# The setup + change-detection + validation steps are shared with the
# source-upload pipeline via templates/steps/common-steps.yml; this template
# appends only the package-build-specific Control Tower call.
# The setup + change-detection + validation steps come from the shared
# templates/steps/common-steps.yml (composed from templates/steps/*); this
# template appends only the package-build-specific Control Tower call.

parameters:
- name: outputDirectory
Expand Down
Loading
Loading