From 3d6ae690841e47f4dc6fc5445a56b530a326cf11 Mon Sep 17 00:00:00 2001 From: "John R. D'Orazio" Date: Wed, 2 Sep 2026 01:09:14 +0200 Subject: [PATCH 1/3] docs(ops): record the staging cutover and correct two Step 1/4 instructions (#955) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The rite-calendar rollout is complete on the dev/staging vhost. Running it surfaced two places where the runbook's instructions do not match how that deployment actually works; both are corrected here alongside the cutover date the runbook asks for. **Step 4 runs itself on a deployed vhost.** `bin/doctrine-migrations` is excluded from the rsync payload precisely because migrations are applied in-process, and `deploy.yaml` POSTs `/_ops/migrate` after every rsync — so `composer db:migrate` cannot run there at all; it fails with `Could not open input file: bin/doctrine-migrations`. The migration landed at deploy time: `Version20260901130000` executed 2026-09-01T20:23:37Z, in the same deploy that shipped #965, which is what the Staging row now records. That also inverts the documented Step 3 → Step 4 order, and the sequence cannot be rearranged, so the runbook now says so rather than leaving an operator to discover it. The effect is to widen the Step 2 window the runbook already describes, not to add a hazard: change-request rows named `rite_calendar` while their tuples were still legacy, so those requests queued for a reviewer instead of auto-approving, and resumed when Step 3 ran. Nothing was corrupted. **Step 1's re-pin command is for a checkout, not the vhost.** `./scripts/setup-openfga.sh --update-env` run from a local clone resolves whatever store that clone points at — a local dev store, not production — and writes those values. The deployed pin against the production store is one file, `api/dev/.env.staging`, the only `.env*` that deployment has. The re-pin shape is documented because getting it wrong took the API down for ~2 minutes on 2026-09-01: a shell redirect truncates its destination BEFORE the command on its left runs, so `cat "$tmp" > .env.staging` where `$tmp` was a `mktemp` file owned by `ubuntu` (mode 600, unreadable to the user the write ran as) emptied the live file and then failed to refill it. phpdotenv reads per request, so a zero-byte `.env.staging` is an immediate outage. `cp` opens its source before truncating its destination and is the right primitive; the whole pipeline must run as the file's owner. Verified end to end against production: model `01M1FHYEDWG8E1CKD4VAKCYZ9V` carries `rite_calendar`, the pin is updated, the tuple migration copied 1 of 1 candidate with 0 skipped, and the copied tuple was confirmed by reading it back from the store rather than trusting the script's report. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_018kGisCz5Gscvc9GGRQAUpz --- docs/ops/rite-calendar-migration-runbook.md | 56 +++++++++++++++++++-- 1 file changed, 51 insertions(+), 5 deletions(-) diff --git a/docs/ops/rite-calendar-migration-runbook.md b/docs/ops/rite-calendar-migration-runbook.md index 45e8ab066..958dc281d 100644 --- a/docs/ops/rite-calendar-migration-runbook.md +++ b/docs/ops/rite-calendar-migration-runbook.md @@ -90,6 +90,31 @@ itself is left in the model unchanged — it is only removed at the prune milest the latest authorization model already in that store (it does not create or upload one), and — with `--update-env` — writes the resulting `OPENFGA_STORE_ID` and `OPENFGA_MODEL_ID` into the `.env.*` file(s) and Docker Compose `.env` files in this repo and its siblings. + + **That command is for a checkout, not for the deployed vhost, and running it in the wrong place + re-pins the wrong store.** The deployed API's pin against the PRODUCTION store lives in exactly + one file — `api/dev/.env.staging` on the VPS, the only `.env*` that deployment has, so there is + no Dotenv precedence to reason about. `api/v4` and `api/v5` carry no `OPENFGA_*` keys at all. + Run from a local checkout the command above resolves whatever store that checkout points at + (a local dev store, not production) and writes those values, so edit the one VPS file instead. + + Edit it as its owner, and land it with `cp` rather than a shell redirect — `cp` opens its + source before truncating its destination, whereas `sed > file` / `cat tmp > file` truncates + first and leaves the file EMPTY if the left-hand command then fails. A zero-byte `.env.staging` + takes the API down immediately, since phpdotenv reads it per request (observed 2026-09-01). For + the same reason, do not build the replacement as a different user: a `mktemp` file made by + `ubuntu` is mode 600 and unreadable to the owner the write runs as. + + ```bash + sudo -u bash -c 'cd \ + && cp -p .env.staging .env.staging.bak.