Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 24 additions & 4 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -134,6 +134,8 @@ jobs:

test-generate-schema:
runs-on: ubuntu-latest
permissions:
contents: read
steps:
- name: Checkout Repository
uses: actions/checkout@v5
Expand All @@ -145,17 +147,35 @@ jobs:

- name: Install Dependencies
run: |
pip install -r requirements-full.txt
pip install jsonschema
python -m pip install -r requirements-full.txt jsonschema pytest

- name: Generate and Test Schema
run: cd schema && python generate_recipe_schema.py

- name: Save Schema as Artifact
- name: Test runtime manifest
run: python -m pytest tests/test_wrangle_runtime_manifest.py -q

- name: Export validated runtime manifest
run: |
python schema/generate_wrangle_manifest.py --source-revision "$GITHUB_SHA" --artifact-dir "$RUNNER_TEMP/runtime-manifest"
cp "$RUNNER_TEMP/runtime-manifest/wrangles-runtime-manifest.json" schema/wrangle_runtime_manifest.json

- name: Save Schema and Runtime Manifest as Artifacts
uses: actions/upload-artifact@v6
with:
name: schema
path: schema/schema.json
path: |
schema/schema.json
schema/wrangle_runtime_manifest.json
schema/wrangles-runtime-manifest.schema.json
if-no-files-found: error

- name: Save runtime manifest bundle as Artifact
uses: actions/upload-artifact@v6
with:
name: runtime-manifest
path: ${{ runner.temp }}/runtime-manifest/
if-no-files-found: error

build:
runs-on: ubuntu-latest
Expand Down
37 changes: 33 additions & 4 deletions .github/workflows/publish-tagged.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ jobs:
contents: read
outputs:
version: ${{ steps.version.outputs.version }}
revision: ${{ steps.version.outputs.revision }}
steps:
- name: Validate deployment app configuration
env:
Expand Down Expand Up @@ -56,6 +57,10 @@ jobs:
id: version
run: |
VERSION="${GITHUB_REF_NAME#v}"
if [[ "$GITHUB_REF_TYPE" != "tag" || ! "$GITHUB_REF_NAME" =~ ^v(0|[1-9][0-9]*)\.(0|[1-9][0-9]*)\.(0|[1-9][0-9]*)$ ]]; then
echo "::error::Production release requires an existing stable vN.N.N tag."
exit 1
fi
SETUP_VERSION=$(sed -n "s/^[[:space:]]*version[[:space:]]*=[[:space:]]*'\([^']*\)'.*/\1/p" setup.py)
if [[ -z "$SETUP_VERSION" ]]; then
echo "::error::Could not read the version from setup.py."
Expand All @@ -66,6 +71,7 @@ jobs:
exit 1
fi
echo "version=${VERSION}" >> "$GITHUB_OUTPUT"
echo "revision=$(git rev-parse HEAD)" >> "$GITHUB_OUTPUT"
echo "Releasing version ${VERSION}"

- name: Validate required GitHub Variables
Expand Down Expand Up @@ -188,6 +194,8 @@ jobs:
test-generate-schema:
runs-on: ubuntu-latest
needs: [validate]
permissions:
contents: read
steps:
- name: Checkout Repository
uses: actions/checkout@v5
Expand All @@ -199,17 +207,38 @@ jobs:

- name: Install Dependencies
run: |
pip install -r requirements-full.txt
pip install jsonschema
python -m pip install -r requirements-full.txt jsonschema pytest

- name: Generate and Test Schema
run: cd schema && python generate_recipe_schema.py

- name: Save Schema as Artifact
- name: Test runtime manifest
run: python -m pytest tests/test_wrangle_runtime_manifest.py -q

- name: Export validated runtime manifest
env:
RELEASE_TAG: ${{ github.ref_name }}
RELEASE_REVISION: ${{ needs.validate.outputs.revision }}
run: |
python schema/generate_wrangle_manifest.py --tag "$RELEASE_TAG" --source-revision "$RELEASE_REVISION" --artifact-dir "$RUNNER_TEMP/runtime-manifest"
cp "$RUNNER_TEMP/runtime-manifest/wrangles-runtime-manifest.json" schema/wrangle_runtime_manifest.json

- name: Save Schema and Runtime Manifest as Artifacts
uses: actions/upload-artifact@v6
with:
name: schema
path: schema/schema.json
path: |
schema/schema.json
schema/wrangle_runtime_manifest.json
schema/wrangles-runtime-manifest.schema.json
if-no-files-found: error

- name: Save runtime manifest bundle as Artifact
uses: actions/upload-artifact@v6
with:
name: runtime-manifest
path: ${{ runner.temp }}/runtime-manifest/
if-no-files-found: error

build:
runs-on: ubuntu-latest
Expand Down
4 changes: 4 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -127,6 +127,7 @@ docs/*
!docs/github-app-deployment.md
!docs/search_ai_mode.md
!docs/search_ai_overview_research.md
!docs/runtime-manifest.md
.DS_Store

# Spyder project settings
Expand All @@ -146,3 +147,6 @@ dmypy.json

# Pyre type checker
.pyre/

# Generated runtime manifest; published as a CI/release artifact
/schema/wrangle_runtime_manifest.json
45 changes: 41 additions & 4 deletions docs/release-lifecycle.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
# Release lifecycle

This document will define how WranglesPY plans, prepares, publishes, verifies,
and closes a release. It is intentionally a stub while the team finalizes the
release process.
This document records the current integration direction and runtime-manifest
artifact steps. The broader planning, ownership and release-closing process is
still being finalized; outstanding decisions are listed below.

## Current integration and deployment direction

Expand All @@ -18,7 +18,7 @@ release process.
transitional automation. It does not publish a Python package or deploy an
environment.
- Create a DEV release deliberately by dispatching `deploy-dev.yml` from
`main`. Create a production release deliberately from a matching `v*` tag on
`main`. Create a production release deliberately from a matching `vN.N.N` tag on
`main`, which invokes `publish-tagged.yml`.

PR #1115 established the current `ci.yml`, `deploy-dev.yml`, and
Expand All @@ -29,6 +29,43 @@ transitional compatibility, not as approval to integrate new work through
tracks the remaining main-SHA validation, secretless PR CI, environment
protection, immutable-artifact, and release-gate work.

## Runtime manifest release steps

Stable production tags use `vN.N.N` matching the literal version in `setup.py`.
The runtime-manifest work for [#1195](https://github.com/wrangleworks/WranglesPY/issues/1195)
extends the generator recovered from `codex/runtime-wrangle-manifest` at commit
`7916bf15`; it is integrated into the current workflow structure.

1. The schema job runs the focused generator tests with the existing Python
3.13 release environment and dependencies from `requirements-full.txt`, plus
`jsonschema` and `pytest`. It verifies the checkout SHA, tag and package version
using the default shallow checkout.
2. It generates `wrangles-runtime-manifest.json` and its `.sha256` sidecar,
then uploads them as the `runtime-manifest` Actions artifact. The original
`schema` artifact also contains the recipe schema, legacy manifest filename
and manifest validation schema.
3. Existing package distribution jobs depend on this validation job. The
manifest is available in that workflow run's Artifacts section. Its presence
alone does not confirm that package publication or deployment succeeded;
select a successful tagged-release run before importing a package contract.
4. If generation or artifact upload fails, rerun the schema job for the same
commit or tag. Reproducing the same bytes requires the same source, Python
and resolved dependency versions; there are no separate manifest constraints.

Manifest delivery currently uses GitHub Actions artifacts only. There is no
manifest publication job for GitHub Releases. Existing package publishing and
Lambda deployment continue through their existing workflows. `ci.yml` also
exports preview manifests for its PR/push runs; `deploy-dev.yml` does not export
runtime manifests for DEV/RC packages in this change. Artifact expiry follows the
repository settings because these uploads do not override `retention-days`.

Automatic download and import into Wrangles-Docs remain a separate part of
[Docs #35](https://github.com/wrangleworks/Wrangles-Docs/issues/35). Before an
import, verify checksum, source version/repository and full tag SHA. Docs also
requires reviewed content updates before advancing from its 1.20.2 pin to 1.20.4;
see [Runtime manifest](runtime-manifest.md) for artifact contents, compatibility
boundaries, required Docs changes and recovery steps.

## Interim milestone rules

- Treat a release milestone as a commitment, not a wishlist.
Expand Down
Loading
Loading