Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
92 changes: 92 additions & 0 deletions .github/workflows/publish.yml
Original file line number Diff line number Diff line change
Expand Up @@ -266,3 +266,95 @@ jobs:
# The tag the publish job verified against protected main, not the raw input.
RELEASE_TAG: ${{ needs.publish.outputs.tag }}
run: gh release upload "${RELEASE_TAG}" "${RUNNER_TEMP}"/installers/* --repo "${GITHUB_REPOSITORY}" --clobber

distribution:
# The release distribution (S3): gentle-pi at the verified tag and the exact
# Pi, with the pnpm-lock.yaml our pinned pnpm resolves for them. It needs the
# npm publication, since gentle-pi@<version> must resolve from the registry.
needs: publish
if: github.repository == 'Gentleman-Programming/gentle-shell'
runs-on: ubuntu-latest
permissions:
contents: read
steps:
- name: Checkout
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6
with:
persist-credentials: false
ref: ${{ github.sha }}

- name: Setup Node.js
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6
with:
node-version: "24"
package-manager-cache: false

- name: Build the distribution assets
env:
# The tag the publish job verified against protected main, not the raw input.
RELEASE_TAG: ${{ needs.publish.outputs.tag }}
run: node scripts/build-distribution.mjs build --shell "${RELEASE_TAG#v}" --out "${RUNNER_TEMP}/distribution"

- name: Keep the assets for verification
uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0
with:
name: gentle-shell-distribution
path: ${{ runner.temp }}/distribution/
if-no-files-found: error
retention-days: 7

distribution-verify:
# The exact assets install with --frozen-lockfile on every system, with our
# pinned Node and pnpm (and Go on Windows), and the launcher runs them.
needs: distribution
if: github.repository == 'Gentleman-Programming/gentle-shell'
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest, macos-latest, windows-latest]
runs-on: ${{ matrix.os }}
timeout-minutes: 45
permissions:
contents: read
steps:
- name: Checkout
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6
with:
persist-credentials: false
ref: ${{ github.sha }}

- name: Setup Node.js
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6
with:
node-version: "24"
package-manager-cache: false

- name: Fetch the distribution assets
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
name: gentle-shell-distribution
path: ${{ runner.temp }}/distribution

- name: Install them frozen and run the launcher
shell: bash
run: node scripts/build-distribution.mjs verify --assets "${RUNNER_TEMP}/distribution"

distribution-assets:
# Attached only after every verify lane passed, with stable names.
needs: [publish, distribution-verify]
if: github.repository == 'Gentleman-Programming/gentle-shell'
runs-on: ubuntu-latest
permissions:
contents: write
steps:
- name: Fetch the verified distribution assets
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
name: gentle-shell-distribution
path: ${{ runner.temp }}/distribution

- name: Attach the distribution assets to the release
env:
GH_TOKEN: ${{ github.token }}
RELEASE_TAG: ${{ needs.publish.outputs.tag }}
run: gh release upload "${RELEASE_TAG}" "${RUNNER_TEMP}/distribution/gentle-shell-distribution.json" "${RUNNER_TEMP}/distribution/gentle-shell-distribution-lock.yaml" --repo "${GITHUB_REPOSITORY}" --clobber
61 changes: 60 additions & 1 deletion docs/bundled-install.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,66 @@ These settings mean:

`installVersion` then runs `node pnpm.mjs install --frozen-lockfile` in the version folder.

After an install that skips build scripts, pnpm 11 rewrites `pnpm-workspace.yaml` in place. It adds each skipped package under `allowBuilds` with the placeholder value `set this to true or false`. The release lockfile (T1b) must therefore be resolved from the unmodified `distributionFiles()` output. It must never come from a `pnpm-workspace.yaml` read back from a folder pnpm already installed into.
After an install that skips build scripts, pnpm 11 rewrites `pnpm-workspace.yaml` in place. It adds each skipped package under `allowBuilds` with the placeholder value `set this to true or false`. The release lockfile is therefore resolved from the unmodified `distributionFiles()` output. It never comes from a `pnpm-workspace.yaml` read back from a folder pnpm already installed into.

## Release distribution assets

Each release publishes two assets with stable names. `scripts/build-distribution.mjs` builds and verifies them.

| Asset | Contents |
|---|---|
| `gentle-shell-distribution.json` | `schema` (1), `shell` (gentle-pi), `pi`, `id` (`<shell>-<pi>`), `generatedWith` (our Node and pnpm pins), `files` (the exact `package.json` and `pnpm-workspace.yaml` that `installVersion` writes) and `lockfile` (`name` and `sha256`). |
| `gentle-shell-distribution-lock.yaml` | The `pnpm-lock.yaml` for those files. |

- **Exact versions.** `shell` is the release version. `pi` is `PI_INSTALL_VERSION` from `scripts/installer-preflight.mjs`, the one Pi pin. A range, a tag such as `latest`, a `v` prefix or build metadata is refused before pnpm runs.
- **Resolution.** `buildDistribution` writes `distributionFiles()` into a fresh folder under the prefix's `tmp/` and runs our pinned pnpm with `install --lockfile-only` in `pnpmEnvironment(layout)`, the environment `installVersion` uses. Only `pnpm-lock.yaml` is read back.
- **Just published.** The registry may not list `gentle-pi@<version>` for a few minutes after `npm publish`. When pnpm reports `ERR_PNPM_NO_MATCHING_VERSION` for exactly that gentle-pi (pnpm prints it on stdout), the resolution is retried in a fresh folder: 10 attempts, 30 seconds apart, about five minutes in all. It then fails with `gentle-pi@<version> is not on the registry after 10 attempts`. Any other failure, including a missing Pi, stops at once.
- **Deterministic.** `distributionAsset(manifest, lockfile)` is plain data in a fixed order, so the same inputs give the same bytes. The lockfile itself records the registry at resolution time, which is why it is built once per release and then only installed.
- **Reading.** `readDistribution(json, lockfile)` returns `{ manifest, lockfile }` only when the versions are exact, the lockfile matches the recorded `sha256`, and `files` equals what this module's `installVersion` writes. `generatedWith` is recorded and not compared.

### One lockfile for every system

pnpm resolves every optional dependency into the lockfile, with its `os`, `cpu` and `libc` fields, whatever system resolves it. Only the install skips the packages for other systems. So no `supportedArchitectures` setting is needed, and none is set.

Observed with pnpm 11.1.1 for gentle-pi 4.0.0 and Pi 1.0.0:

- A lockfile resolved on macOS arm64 lists every `@yuuang/ffi-rs-*`, `@ff-labs/fff-bin-*` and `@esbuild/*` variant for Linux (gnu and musl), Windows and macOS.
- A `--frozen-lockfile` install of that lockfile on macOS with `supportedArchitectures` forced to Linux and Windows installed the Linux and Windows variants. The lockfile was left unchanged.
- The regular frozen install on macOS installed only the darwin-arm64 variants.

### Minimum release age

pnpm 11.1.1 defaults `minimum-release-age` to 1440 minutes (one day), in non-strict mode. The default is kept on purpose, and no value is set:

- **Ranged dependencies.** For a ranged transitive dependency, pnpm picks the highest matching version that is at least one day old. A version published minutes before a release does not enter the lockfile. This is a supply-chain guard.
- **Exact pins.** gentle-pi and Pi are exact. In non-strict mode, when no matching version is old enough, pnpm falls back to the matching version anyway (`pickRespectingMinReleaseAge`). So a gentle-pi published minutes ago still resolves.
- **Never set it explicitly.** pnpm 11.1.1 turns `minimumReleaseAgeStrict` on whenever `minimumReleaseAge` is set explicitly and the strict flag is not. A strict check would refuse the gentle-pi the release just published.

Observed with pnpm 11.1.1:

- `pnpm_config_minimum_release_age=5256000` alone failed.
- The same value with `pnpm_config_minimum_release_age_strict=false` resolved gentle-pi 4.0.0 exactly.
- The default resolved it too.

`pnpmEnvironment` drops every inherited `pnpm_*` variable, so a host setting cannot change this.

### Release workflow

`.github/workflows/publish.yml` runs three jobs after `publish`, because `gentle-pi@<version>` must resolve from npm first:

1. **`distribution`** checks out the verified release commit. It runs `build-distribution.mjs build --shell <tag without v> --out <dir>` with our pinned Node and pnpm in a temporary prefix, and keeps both files as a workflow artifact.
2. **`distribution-verify`** runs on `ubuntu-latest`, `macos-latest` and `windows-latest`. It runs `build-distribution.mjs verify --assets <dir>`, which:
- downloads our pinned Node and pnpm into a temporary prefix (with Go on Windows);
- installs the exact pair with `installVersion` (`--frozen-lockfile`);
- switches `current` and writes the launcher;
- requires `gentle-shell --version` to report exactly `gentle-shell <shell>` and `pi <pi>`.

On Windows a fresh temporary folder stands in for the private-folder claim, which has its own native tests. The launcher runs in `pnpmEnvironment`, the environment our pnpm ran in, without any `PI_*` or `GENTLE_*` variable. So no host `NODE_OPTIONS`, `NODE_PATH`, npm or pnpm setting, or Pi override reaches it.
3. **`distribution-assets`** runs only after every verify lane passed. It attaches both files to the release with `gh release upload --clobber`.

Both `build` and `verify` create their temporary prefix (`gsd-*` in the system temporary folder) only after their arguments are accepted. They always remove it, whether the command succeeded or failed.

The release gets the same bytes that the three systems installed.

## npm is not needed

Expand Down
Loading
Loading