Release 0.16.0

.github/actions/setup-mkosi/action.yml

@@ -0,0 +1,38 @@
+# Set up an ubuntu-24.04 runner to build a moose image with mkosi. Shared by
+# ci-cloud-image.yml (the build, boot proofs and publish) and os-lock-bump.yml
+# (the scheduled OS package lock bump, #560), so both build with the same mkosi
+# and the same host tools, and a bump resolves exactly what CI will check.
+name: Set up mkosi
+description: Relax the userns restriction, install the image-build tools and mkosi v26.
+runs:
+  using: composite
+  steps:
+    # mkosi 26 builds rootless: it unshares a user namespace and drops
+    # capabilities inside it. Ubuntu 24.04 ships
+    # kernel.apparmor_restrict_unprivileged_userns=1, which hands an unconfined
+    # process a userns with no CAP_SETPCAP, so mkosi's PR_CAPBSET_DROP returns
+    # EPERM and the build dies at sandbox bring-up (#189). Relaxing the knob is
+    # safe on an ephemeral runner.
+    - name: Relax AppArmor unprivileged-userns restriction
+      shell: bash
+      run: |
+        knob=/proc/sys/kernel/apparmor_restrict_unprivileged_userns
+        echo "before: $(cat "$knob" 2>/dev/null || echo 'n/a')"
+        sudo sysctl -w kernel.apparmor_restrict_unprivileged_userns=0
+        echo "after:  $(cat "$knob")"
+
+    # Host tooling mkosi needs to bootstrap a Debian tools tree and build
+    # rootless (uidmap/mmdebstrap/keyring; dosfstools/mtools for the ESP;
+    # squashfs-tools/e2fsprogs for the root; systemd-container for the nspawn
+    # build sandbox; libpam0g-dev for the slim host-agent-real CGO build).
+    - name: Install mkosi and image-build tooling
+      shell: bash
+      run: |
+        sudo apt-get update
+        sudo apt-get install -y \
+          uidmap mmdebstrap debian-archive-keyring \
+          dosfstools mtools squashfs-tools e2fsprogs systemd-container \
+          libpam0g-dev
+        git clone --depth=1 --branch v26 \
+          https://github.com/systemd/mkosi.git /tmp/mkosi-src
+        sudo ln -sf /tmp/mkosi-src/bin/mkosi /usr/local/bin/mkosi

.github/workflows/ci-go.yml

@@ -15,14 +15,18 @@ name: CI / Go
 
 on:
   pull_request:
-    branches: [dev, main]
+    # hotfix/** gates an OS lock bump into a patch-release branch (#560).
+    branches: [dev, main, "hotfix/**"]
     paths:
       - "**.go"
       - "go.mod"
       - "go.sum"
       - "Makefile"
       - "VERSION"
       - "api/**"
+      # The lock files have no .go in them, but dev/os-lock/oslock's test checks
+      # they fit together, so a lock bump PR must run it (#560).
+      - "dev/os-lock/**"
       - ".github/workflows/ci-go.yml"
 
 # Cancel superseded runs on the same ref (e.g. force-pushes to a PR branch).

CLAUDE.md

@@ -12,7 +12,7 @@ Two phrases that constrain a lot of design:
 A running moose is five processes/artifacts. Three are Go, one is JavaScript, one is a container we don't write.
 
 - **`moose-brain`** (`cmd/brain/`, `internal/`) — the control-plane daemon. One Go binary: owns SQLite state, the REST+SSE API, the app lifecycle, and the Caddy config. Drives Docker via the `docker compose` CLI.
-- **`host-agent`** — the privileged side. Two binaries. `cmd/host-agent/` is the **fake** used by the inner dev loop: it speaks the real `BRAIN_HOST_PROTOCOL.md` wire format over a real UNIX socket, but the host ops are stubbed in memory. `cmd/host-agent-real/` is the real one, and most of it works now: PAM verify, user management, `/proc` sampling, disk and RAM, journal streaming, service health, reboot-required, time zone, Avahi discovery, the first-boot brain launch, and the control-plane update path. Not wired yet: LUKS/TPM, apt, and NetworkManager config. It also builds a slim `hosted` variant (`go build -tags hosted`) for the cloud image. See `docs/architecture.md` # Components for the current split.
+- **`host-agent`** — the privileged side. Two binaries. `cmd/host-agent/` is the **fake** used by the inner dev loop: it speaks the real `BRAIN_HOST_PROTOCOL.md` wire format over a real UNIX socket, but the host ops are stubbed in memory. `cmd/host-agent-real/` is the real one, and most of it works now: PAM verify, user management, `/proc` sampling, disk and RAM, journal streaming, service health, reboot-required, time zone, Avahi discovery, the first-boot brain launch, and the control-plane update path. Not wired yet: LUKS/TPM, NetworkManager config, and the A/B OS update (#486, designed in `UPDATES.md` # 1 and `BUILD.md` # 1b; there is no apt on the update path). It also builds a slim `hosted` variant (`go build -tags hosted`) for the cloud image. See `docs/architecture.md` # Components for the current split.
 - **`web-ui`** (`web-ui/`) — Vue 3 + Vite + TanStack Query dashboard. Talks only to the brain.
 - **Caddy** (`dev/`) — reverse proxy. Terminates `*.local` and routes to app containers + the brain, configured live by the brain via Caddy's admin API. Subdomain routing per app, never path-based (browser same-origin policy is the reason).
 - **SQLite** — the brain's only persistent store (`internal/store/`).
@@ -61,7 +61,7 @@ The inner/outer boundary is also the **cross-platform / Linux-only** boundary. T
 - `make test-nopam` — full suite minus the PAM package, for when you don't have `libpam0g-dev` (i.e. off Linux).
 - `make clean` — stop dev Caddy, remove moose containers/networks, wipe `.dev/state`.
 
-**Building the hosted cloud image is a CI job: don't build it locally.** `make build-cloud-image` / `make test-cloud-qemu` need root + `/dev/kvm` + mkosi and take ~10+ min; a local mkosi build is fragile and easy to get wrong (a broken build is what once produced a phantom "`:443` doesn't bind" hunt). Instead trigger the **`CI / Cloud image`** GitHub Action: `gh workflow run "CI / Cloud image" --ref <branch> -f publish=false` builds the image and runs the QEMU boot-proof (`unseeded seeded bios access update ssh remap` boots; add `-f boots="remap"` to run only some) **without** publishing anything. Only `publish=true` (the default) publishes anything. Since #352 that means two things: it **attaches the compressed image and its checksum to the tagged GitHub Release**, and it **pushes the brain and UI images to ghcr**. It uploads to no hosting provider and holds no provider login. Publishing is a deliberate act, not a test. See [`docs/dev/hosted-boot-proof.md`](docs/dev/hosted-boot-proof.md) for reading the result and debugging a red boot.
+**Building the hosted cloud image is a CI job: don't build it locally.** `make build-cloud-image` / `make test-cloud-qemu` need root + `/dev/kvm` + mkosi and take ~10+ min; a local mkosi build is fragile and easy to get wrong (a broken build is what once produced a phantom "`:443` doesn't bind" hunt). Instead trigger the **`CI / Cloud image`** GitHub Action: `gh workflow run "CI / Cloud image" --ref <branch> -f publish=false` builds the image and runs the QEMU boot-proof (`unseeded seeded bios access update ssh remap` boots; add `-f boots="remap"` to run only some) **without** publishing anything. Only `publish=true` (the default) publishes anything. Since #352 that means two things: it **attaches the compressed image and its checksum to the tagged GitHub Release**, and it **pushes the brain and UI images to ghcr**. Since #559 those are two release lines (the OS and the control plane), and dispatch also takes `publish_os` and `publish_control_plane` to publish just one of them, for example `-f publish=false -f publish_control_plane=true` to re-publish only the control plane. A published file or version tag is never overwritten; only `latest` moves, and only forward (`BUILD.md` # 6). It uploads to no hosting provider and holds no provider login. Publishing is a deliberate act, not a test. See [`docs/dev/hosted-boot-proof.md`](docs/dev/hosted-boot-proof.md) for reading the result and debugging a red boot.
 
 **Prerequisites for the inner loop:** Docker + `docker compose`, Node 20+, Go 1.23+, host port `:80` free (dev Caddy binds it so `<slug>.local` works portless), and `avahi-daemon` running on Linux (so `.local` names resolve under `make dev`). The full Go test suite additionally needs `libpam0g-dev` on Linux; see `docs/dev/running-locally.md`.
 
@@ -92,7 +92,7 @@ Small set of rules. Codified now so we don't have to back them out later.
 - **Consumer-side interfaces.** Interfaces live in the package that *uses* them, not the package that implements them. `lifecycle.DockerDriver` lives in `internal/lifecycle/`, not in a hypothetical `internal/docker/`. Provider packages export concrete types only. Exception: a single interface shared by three or more consumers can move to the provider, but default to consumer-side until that's true.
 - **Layer boundaries.** `internal/lifecycle` is the transaction owner; only `cmd/brain` and `internal/api` may import it. `internal/store` is the persistence boundary; only `internal/lifecycle`, `internal/api`, `internal/auth`, `internal/audit`, and `cmd/brain` may import it. Anything else reaching in is breaking the model — push the call through the right seam instead.
 - **`log/slog` is the only logger.** No `"log"` imports, no `fmt.Println` for diagnostics. Structured fields, not interpolated strings: `slog.Info("app installed", "instance_id", id)`, not `slog.Info(fmt.Sprintf("installed %s", id))`. The default handler is set in `cmd/brain/main.go`; use `slog.Default()` (the package-level functions) — don't thread `*slog.Logger` through constructors.
-- **Standard structured fields.** Use these key names so journalctl/jq filters stay stable: `instance_id`, `manifest_id`, `slug`, `service`, `image`, `host`, `upstream`, `step`, `err`, `output`, `user_id`, `username`, `role`, `action`, `actor_user_id`, `target_kind`, `target_id`, `retry_after`, `retry_after_s`, `name`, `uid`, `iface`, `interfaces`, `src`, `dir`, `profile`, `box_id`, `zone`, `exposure`, `trusted_proxies`, `brain`, `ui`, `minimum_host_agent`, `keys`, `state_dir`, `job_id`, `window`, `url`, `from`, `count`, `dropped`, `tier`, `remap_base`, `gid`, `image_user`. `host` is a machine or upstream hostname only — a single network interface name is `iface`, a list of them is `interfaces` (never overload `host` for either). `src` is a source filesystem path (bind-source, folder-source); `dir` is a relative bind dir path. `profile` is the resolved environment profile (`appliance`|`hosted`, `ENVIRONMENT.md`). `box_id` is the hosted box's provisioned identity (`ENVIRONMENT.md` # Provisioning). `zone` is an IANA time-zone name (`TIME.md`, host-agent set-timezone). `exposure` is an app's per-instance access mode (`restricted`|`public`, `ENVIRONMENT.md` #306). `trusted_proxies` is the configured set of proxies whose `X-Forwarded-For` the brain reads when deriving a client IP (`BRAIN_UI_PROTOCOL.md` # Rate limiting & abuse). `brain` and `ui` are the two control-plane versions a release names, and `minimum_host_agent` the host-agent version it requires (`RELEASE_MANIFEST.md`); use them for versions, not for image refs — an image ref is `image`. `keys` is how many signing keys a build accepts, and `state_dir` a state directory path (`src` stays for a source path being read or bound). `job_id` is a host-agent job id (`internal/hostagent/jobs.go`), and `window` the configured update window (`UPDATES.md` # 8.4). `url` is an HTTP endpoint the box reads, and `from` says which of several configured sources a setting came from, one of `answer` (the control plane's update-target answer), `seed`, `env`, `default` (`UPDATES.md` # 8.4); where there are only two sources, the boolean `from_<source>` form is used instead (`from_ledger`). `dropped` lists what the box read but could not use, from catalog data it reads leniently (AI provider entries, manifest `role` and `requires` keys), and `count` is how many there were. `tier` is an instance's user-namespace tier (`default`|`caps`|`image`|`host`, `APP_ISOLATION.md` # User-namespace tiers), and `remap_base` the first host id of the Docker remap range (0 for none). `image_user` is the user an image sets (its `Config.User`, a name or a number, as the image wrote it), which the image tier resolves to ids. `name` is an app instance's display name (it rides alongside `instance_id`, never instead of it), and `uid` a numeric Unix user id — the allocated app-service identity, a resolved home owner, or an image user's id. `gid` is its numeric group id. `retry_after` and `retry_after_s` come from the two throttles that `AUTH.md` # Rate limiting keeps apart on purpose. Both are correct. Do not merge them. `retry_after` is the login backoff's wait, written as a duration string (`internal/api/auth.go`); that path sends no `Retry-After` header, by design. `retry_after_s` is the general request limiter's wait, written as a whole number of seconds; it matches that limiter's `retry_after_s` JSON field and the `Retry-After` header it sets (`internal/api/ratelimit.go`, `BRAIN_UI_PROTOCOL.md` # 429 contract). To find every throttled request you must search for both keys. That is the price of the split, not a bug. Adding a new recurring field? Add it here.
+- **Standard structured fields.** Use these key names so journalctl/jq filters stay stable: `instance_id`, `manifest_id`, `slug`, `service`, `image`, `host`, `upstream`, `step`, `err`, `output`, `user_id`, `username`, `role`, `action`, `actor_user_id`, `target_kind`, `target_id`, `retry_after`, `retry_after_s`, `name`, `uid`, `iface`, `interfaces`, `src`, `dir`, `profile`, `box_id`, `zone`, `exposure`, `trusted_proxies`, `brain`, `ui`, `minimum_host_agent`, `keys`, `state_dir`, `job_id`, `window`, `url`, `from`, `count`, `dropped`, `tier`, `remap_base`, `gid`, `image_user`, `os`, `slot`, `digest`. `host` is a machine or upstream hostname only — a single network interface name is `iface`, a list of them is `interfaces` (never overload `host` for either). `src` is a source filesystem path (bind-source, folder-source); `dir` is a relative bind dir path. `profile` is the resolved environment profile (`appliance`|`hosted`, `ENVIRONMENT.md`). `box_id` is the hosted box's provisioned identity (`ENVIRONMENT.md` # Provisioning). `zone` is an IANA time-zone name (`TIME.md`, host-agent set-timezone). `exposure` is an app's per-instance access mode (`restricted`|`public`, `ENVIRONMENT.md` #306). `trusted_proxies` is the configured set of proxies whose `X-Forwarded-For` the brain reads when deriving a client IP (`BRAIN_UI_PROTOCOL.md` # Rate limiting & abuse). `brain` and `ui` are the two control-plane versions a release names, and `minimum_host_agent` the host-agent version it requires (`RELEASE_MANIFEST.md`); use them for versions, not for image refs — an image ref is `image`. `keys` is how many signing keys a build accepts, and `state_dir` a state directory path (`src` stays for a source path being read or bound). `job_id` is a host-agent job id (`internal/hostagent/jobs.go`), and `window` the configured update window (`UPDATES.md` # 8.4). `url` is an HTTP endpoint the box reads, and `from` says which of several configured sources a setting came from, one of `answer` (the control plane's update-target answer), `seed`, `env`, `default` (`UPDATES.md` # 8.4); where there are only two sources, the boolean `from_<source>` form is used instead (`from_ledger`). `dropped` lists what the box read but could not use, from catalog data it reads leniently (AI provider entries, manifest `role` and `requires` keys), and `count` is how many there were. `tier` is an instance's user-namespace tier (`default`|`caps`|`image`|`host`, `APP_ISOLATION.md` # User-namespace tiers), and `remap_base` the first host id of the Docker remap range (0 for none). `image_user` is the user an image sets (its `Config.User`, a name or a number, as the image wrote it), which the image tier resolves to ids. `os` is an OS (moose) release version, as `brain` and `ui` are control-plane versions; `slot` is an A/B OS slot (`A`|`B`, `BUILD.md` # 1b); `digest` is an OS bundle's sha256 (`UPDATES.md` # 1). `name` is an app instance's display name (it rides alongside `instance_id`, never instead of it), and `uid` a numeric Unix user id — the allocated app-service identity, a resolved home owner, or an image user's id. `gid` is its numeric group id. `retry_after` and `retry_after_s` come from the two throttles that `AUTH.md` # Rate limiting keeps apart on purpose. Both are correct. Do not merge them. `retry_after` is the login backoff's wait, written as a duration string (`internal/api/auth.go`); that path sends no `Retry-After` header, by design. `retry_after_s` is the general request limiter's wait, written as a whole number of seconds; it matches that limiter's `retry_after_s` JSON field and the `Retry-After` header it sets (`internal/api/ratelimit.go`, `BRAIN_UI_PROTOCOL.md` # 429 contract). To find every throttled request you must search for both keys. That is the price of the split, not a bug. Adding a new recurring field? Add it here.
 - **Typed errors at boundaries, not everywhere.** Define a sentinel/typed error only when a *consumer* needs to discriminate (HTTP status, retry decision, UI text). `store.ErrNotFound` exists because the API maps it to 404. Don't pre-declare error types speculatively.
 - **No premature abstraction.** Don't introduce an interface, factory, or DI container until at least two concrete consumers exist. It bites hardest in Go where every extra interface is import-graph weight.
 - **`internal/` for everything except `cmd/`.** No `pkg/`. Anything inside `internal/` is private to this module by Go's own rules — no public API surface to maintain.

CONTROL_PLANE_VERSION

@@ -0,0 +1 @@
+0.16.0