diff --git a/CHANGELOG.md b/CHANGELOG.md index 60ccfcf..0fdf5ec 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -10,6 +10,7 @@ Changes are grouped by date. ### Changed +- `spark-http-proxy` skill: the certificate commands are documented as the `certs` topic (`list`, `describe`, `generate`, `delete`); `certs describe` is the first step on a certificate warning, and the deprecated `generate-mkcert`, `list-certs` and `remove-cert` warnings are explained as expected - `security-assessment` and `agentic-security-audit` skills: moved from `skills/system/` to a new optional `security` category. They are no longer installed by default; enable them with `ajust sf-harness-category enable security` or `sjust sf-harness-category enable security` ## [2026-09-01] diff --git a/skills/system/spark-http-proxy/SKILL.md b/skills/system/spark-http-proxy/SKILL.md index a58c436..f3a42f3 100644 --- a/skills/system/spark-http-proxy/SKILL.md +++ b/skills/system/spark-http-proxy/SKILL.md @@ -56,7 +56,7 @@ picture and how to update. | The user wants to… | Do this | Read | | ------------------------------------------------ | ------------------------------ | -------------------------------- | | Make a container reachable at a domain | Edit its `compose.yml` service | `references/expose-container.md` | -| Get trusted HTTPS / fix certificate warnings | Run `generate-mkcert` | `references/certificates.md` | +| Get trusted HTTPS / fix certificate warnings | Run `certs generate` | `references/certificates.md` | | Resolve `*.loc` (or other TLDs) on their machine | Run `configure-dns` | `references/dns.md` | | Fix "it's not working / not reachable" | Walk the decision tree | `references/troubleshooting.md` | | Understand how it's installed / update it | Per-platform provisioner notes | `references/provisioning.md` | @@ -65,7 +65,7 @@ picture and how to update. Dependencies: Docker is required (the proxy is a Docker stack; `spark-http-proxy self-test` checks the daemon). `mkcert` is only needed for trusted HTTPS and is -installed automatically by `generate-mkcert` — you rarely check it yourself. See +installed automatically by `certs generate`, so you rarely check it yourself. See `references/uninstall.md` for how to verify what is installed, where certificates and config live, and how to uninstall. @@ -139,23 +139,32 @@ For multiple domains, wildcards/regex, the full `VIRTUAL_PATH` rules, the native HTTPS works out of the box with a self-signed certificate (browser warning). For a trusted certificate, generate one with mkcert — it installs mkcert if needed, -writes the cert, and restarts Traefik: +writes the cert, and applies it to the running proxy without a restart: ```bash # Wildcard for the SparkFabrik convention: covers myapp.spark.loc, api.spark.loc, … -spark-http-proxy generate-mkcert "*.spark.loc" +spark-http-proxy certs generate "*.spark.loc" # Or a specific host -spark-http-proxy generate-mkcert "myapp.spark.loc" +spark-http-proxy certs generate "myapp.spark.loc" ``` The key gotcha: a wildcard covers exactly one label level, so match it to the level directly above the host. `*.spark.loc` covers `myapp.spark.loc` but **not** -a deeper host like `drupal.client.spark.loc` — that needs `*.client.spark.loc`. +a deeper host like `drupal.client.spark.loc`, which needs `*.client.spark.loc`. And `*.loc` does **not** cover `myapp.spark.loc` at all, which is why the -convention's base certificate is `*.spark.loc`, not `*.loc`. Full details, the -cert directory, SNI matching, and manual generation are in -`references/certificates.md`. +convention's base certificate is `*.spark.loc`, not `*.loc`. When a user reports a +warning on a hostname, run `spark-http-proxy certs describe ` first: it +names the certificate that covers it, or says which wildcard falls one label +short and which one to generate. `certs list` shows every installed certificate +with its files. Full details, `delete`, SNI matching, and manual generation are +in `references/certificates.md`. + +The certificate commands were `generate-mkcert`, `list-certs` and `remove-cert` +until September 2026. Those names still run and print +`generate-mkcert is deprecated, use: spark-http-proxy certs generate` on stderr. +That warning is expected on an up-to-date CLI, not a broken install; use the +`certs` form in anything you write. ## Guiding a user @@ -163,21 +172,22 @@ When the user just wants to understand the tool rather than have you change files, run `spark-http-proxy help` for the authoritative command list and explain the relevant commands. The lifecycle and utility commands: -| Command | Purpose | -| ----------------------------------------- | ------------------------------------------------------ | -| `start` / `start-with-metrics` | Start the proxy (optionally with Prometheus/Grafana) | -| `status` | Show running services and the dashboard URL | -| `hosts [describe ]` | What is served, by which machine, from which directory | -| `restart` / `stop-metrics` | Restart the stack / stop only monitoring | -| `start-with-tailscale` / `stop-tailscale` | Start with, or stop, tailnet peer routing | -| `tailscale-peers [--refresh]` | Show the last discovery cycle, or run one first | -| `generate-mkcert ` | Create trusted certificates for a domain | -| `configure-dns` | Wire system DNS to resolve the proxy TLDs | -| `show-config` | Print current configuration and file locations | -| `logs [service]` | Tail logs (optionally for one service) | -| `dashboard` / `grafana` / `prometheus` | Open the respective web UI | -| `upgrade` / `self-update` | Update images / update the script and compose files | -| `clean` / `destroy` | Stop + remove volumes / remove everything | +| Command | Purpose | +| ------------------------------------------ | ------------------------------------------------------------------- | +| `start` / `start-with-metrics` | Start the proxy (optionally with Prometheus/Grafana) | +| `status` | Show running services and the dashboard URL | +| `hosts [describe ]` | What is served, by which machine, from which directory | +| `restart` / `stop-metrics` | Restart the stack / stop only monitoring | +| `start-with-tailscale` / `stop-tailscale` | Start with, or stop, tailnet peer routing | +| `tailscale-peers [--refresh]` | Show the last discovery cycle, or run one first | +| `certs list` / `certs describe ` | Installed certificates, or what one covers and whether it is served | +| `certs generate` / `certs delete ` | Create, or remove, trusted certificates for a domain | +| `configure-dns` | Wire system DNS to resolve the proxy TLDs | +| `show-config` | Print current configuration and file locations | +| `logs [service]` | Tail logs (optionally for one service) | +| `dashboard` / `grafana` / `prometheus` | Open the respective web UI | +| `upgrade` / `self-update` | Update images / update the script and compose files | +| `clean` / `destroy` | Stop + remove volumes / remove everything | Behavior is tuned with env vars, most usefully `HTTP_PROXY_DNS_TLDS` (default `loc`) to serve additional TLDs such as `dev`. See `references/dns.md`. diff --git a/skills/system/spark-http-proxy/references/certificates.md b/skills/system/spark-http-proxy/references/certificates.md index b5eccac..8e10beb 100644 --- a/skills/system/spark-http-proxy/references/certificates.md +++ b/skills/system/spark-http-proxy/references/certificates.md @@ -5,27 +5,91 @@ automatically. Without a matching certificate, Traefik serves a self-signed one and the browser shows a warning. To make local HTTPS trusted, generate a certificate with mkcert. +## The `certs` commands + +All certificate work is under one topic. `spark-http-proxy certs help` prints it. + +| Command | Does | +| -------------------------- | -------------------------------------------------------------------- | +| `certs list` | Table of installed certificates and the files holding each one | +| `certs describe ` | What one certificate covers, its dates, issuer, whether it is served | +| `certs generate ` | Generate with mkcert and apply to the running proxy, no restart | +| `certs delete ...` | Remove certificate and key, one confirmation, apply to the proxy | + +The former names `generate-mkcert`, `list-certs` and `remove-cert` still run and +print a deprecation warning on stderr naming the replacement. Seeing +`list-certs is deprecated, use: spark-http-proxy certs list` in a user's terminal +is expected on an up-to-date CLI, not a sign of a broken install. Write the +`certs` form. + ## Generate with the CLI (preferred) ```bash # SparkFabrik convention: covers myapp.spark.loc, api.spark.loc, … -spark-http-proxy generate-mkcert "*.spark.loc" +spark-http-proxy certs generate "*.spark.loc" # A specific host -spark-http-proxy generate-mkcert "myapp.spark.loc" +spark-http-proxy certs generate "myapp.spark.loc" # A deeper level, if a project nests further (see the gotcha below) -spark-http-proxy generate-mkcert "*.project.spark.loc" +spark-http-proxy certs generate "*.project.spark.loc" ``` -`generate-mkcert` does everything: installs mkcert if missing (Homebrew on macOS, +`certs generate` does everything: installs mkcert if missing (Homebrew on macOS, pacman on Arch; on other Linux distros it prints manual install steps), runs `mkcert -install` to add the local CA to the system trust store, creates the certificate directory `~/.local/spark/http-proxy/certs`, writes the cert with a -safe filename, and **restarts Traefik** so it loads immediately. No config file -editing is needed — the Traefik entrypoint scans the certs directory and -generates the TLS config (`/traefik/dynamic/auto-tls.yml`). To check or remove -mkcert, the certs, or the CA, see `uninstall.md`. +safe filename, and applies it to the running proxy **without a restart**, so +nothing else the proxy serves drops a connection. No config file editing is +needed: the Traefik entrypoint scans the certs directory and generates the TLS +config (`/traefik/dynamic/auto-tls.yml`). To check or remove mkcert, the certs, +or the CA, see `uninstall.md`. + +## List, describe, delete + +`certs list` is the inventory. The directory is printed once, then one row per +certificate. A wildcard is stored as `_wildcard_`, and a key that is not beside +its certificate shows as `missing`. It needs neither Docker nor the proxy: + +```text +Certificates in ~/.local/spark/http-proxy/certs + +DOMAIN CERTIFICATE KEY +api.spark.loc api.spark.loc.pem api.spark.loc-key.pem +*.spark.loc _wildcard_.spark.loc.pem _wildcard_.spark.loc-key.pem + +2 certificates. Remove one with: spark-http-proxy certs delete '*.spark.loc' +``` + +`certs describe` reads one certificate with openssl. Give it a hostname rather +than a certificate name and it finds the certificate covering it, or says why +none does. This is the first command to run on a certificate warning: + +```bash +spark-http-proxy certs describe "*.spark.loc" # the certificate itself +spark-http-proxy certs describe "app.spark.loc" # covered by *.spark.loc, shows it +spark-http-proxy certs describe "a.b.spark.loc" # not covered, explains why +``` + +```text +*.spark.loc + certificate ~/.local/spark/http-proxy/certs/_wildcard_.spark.loc.pem + private key ~/.local/spark/http-proxy/certs/_wildcard_.spark.loc-key.pem + covers *.spark.loc + valid 2025-07-13 to 2027-10-13 + issued by mkcert paolo@workstation + served yes, by the running proxy +``` + +`served` is read from the running proxy; with the proxy stopped it says so. If +`describe` stops with a message about openssl, that machine lacks a usable +`openssl` (sparkdock installs it as `openssl@3`); `list`, `generate` and `delete` +do not need it. + +`certs delete` takes the same domains that were passed to `generate`, wildcards +included, lists every match, reports any it cannot find, asks once, and applies +the change to the running proxy. It refuses to delete without a terminal to +confirm on, so it cannot be scripted into silently removing certificates. ## The wildcard nesting gotcha @@ -44,12 +108,13 @@ Rule of thumb: for `.`, generate `*.`. So a per-client nested scheme like `..spark.loc` needs `*..spark.loc`: ```bash -spark-http-proxy generate-mkcert "*.client.spark.loc" # covers drupal.client.spark.loc, api.client.spark.loc, … +spark-http-proxy certs generate "*.client.spark.loc" # covers drupal.client.spark.loc, api.client.spark.loc, … ``` If a user reports a certificate warning on a nested name while a broader wildcard -exists, this mismatch is almost always the cause: generate the wildcard one level -above the host. +exists, this mismatch is almost always the cause. `spark-http-proxy certs +describe ` confirms it: it names the wildcard that falls one label +short and prints the `certs generate` command for the one that would cover it. ## How matching works (SNI) @@ -65,8 +130,8 @@ sits on and needs none of its own. Passing a path to the certificate commands is refused: ```bash -spark-http-proxy generate-mkcert myapp.spark.loc # correct -spark-http-proxy generate-mkcert myapp.spark.loc/api # refused +spark-http-proxy certs generate myapp.spark.loc # correct +spark-http-proxy certs generate myapp.spark.loc/api # refused ``` ## Manual generation (alternative) @@ -79,8 +144,9 @@ mkdir -p ~/.local/spark/http-proxy/certs mkcert -cert-file ~/.local/spark/http-proxy/certs/wildcard.spark.loc.pem \ -key-file ~/.local/spark/http-proxy/certs/wildcard.spark.loc-key.pem \ "*.spark.loc" -docker compose restart # reload certs (CLI does this for you) +spark-http-proxy restart # a hand-written cert is applied on restart ``` The cert and key go in `~/.local/spark/http-proxy/certs`, which Traefik mounts -read-only. After dropping files there manually, restart the proxy so they load. +read-only. After dropping files there manually, restart the proxy so they load; +certificates made with `certs generate` are applied without one. diff --git a/skills/system/spark-http-proxy/references/peer-routing.md b/skills/system/spark-http-proxy/references/peer-routing.md index d8e8655..9a09e74 100644 --- a/skills/system/spark-http-proxy/references/peer-routing.md +++ b/skills/system/spark-http-proxy/references/peer-routing.md @@ -97,7 +97,7 @@ subject: CN=TRAEFIK DEFAULT CERT Fix it on the machine doing the reaching, not the one serving: ```bash -spark-http-proxy generate-mkcert 'macos.test.spark.loc' +spark-http-proxy certs generate 'macos.test.spark.loc' ``` **A wildcard covers exactly one label.** This is the part that catches people, so diff --git a/skills/system/spark-http-proxy/references/provisioning.md b/skills/system/spark-http-proxy/references/provisioning.md index 17c13bb..6029a78 100644 --- a/skills/system/spark-http-proxy/references/provisioning.md +++ b/skills/system/spark-http-proxy/references/provisioning.md @@ -9,7 +9,7 @@ platform, and they differ in important ways — especially the DNS target IP. | Installs CLI | git clone to `/opt/sparkdock/http-proxy` + symlink `/usr/local/bin/spark-http-proxy` | `get_url` of `bin/spark-http-proxy` → `/usr/local/bin/spark-http-proxy` (0755); no clone | | Compose file | from the clone | `bin/compose.yml` → `~/.local/spark/http-proxy/compose.yml` | | Starts the proxy | **yes** (`spark-http-proxy start`) | **no** — run `spark-http-proxy start` yourself | -| mkcert local CA | package only; CA trust via `sjust system-install-mkcert` or `generate-mkcert` | `mkcert -install` **is** run during provisioning | +| mkcert local CA | package only; CA trust via `sjust system-install-mkcert` or `certs generate` | `mkcert -install` **is** run during provisioning | | `*.loc` DNS | `/etc/resolver/loc` → `127.0.0.1` port `19322` | systemd-resolved drop-in → `127.0.0.1:19322` (Arch and Debian/Ubuntu) | | CLI guaranteed | yes (fail-fast verification) | no explicit verification step | @@ -32,7 +32,7 @@ interfaces. What differs is the mechanism, not the target. `spark-http-proxy start` — the user does that. If `*.loc` still does not resolve on a Linux box, run `configure-dns`. - **mkcert CA trust differs.** Linux provisioning trusts the CA (`mkcert -install`) - automatically; macOS does not until `generate-mkcert` or `sjust + automatically; macOS does not until `certs generate` or `sjust system-install-mkcert` runs. A lingering TLS warning on macOS often means that step has not run. diff --git a/skills/system/spark-http-proxy/references/troubleshooting.md b/skills/system/spark-http-proxy/references/troubleshooting.md index c50e5f1..6207f8b 100644 --- a/skills/system/spark-http-proxy/references/troubleshooting.md +++ b/skills/system/spark-http-proxy/references/troubleshooting.md @@ -97,7 +97,9 @@ The symptom here is a **browser TLS trust warning** ("not secure", `NET::ERR_CER on a page that otherwise loads — not a 404 and not a refused connection. If the page does not load at all, it is a routing or DNS problem above, not this. -1. Trusted cert never generated → run `spark-http-proxy generate-mkcert "*.spark.loc"`. +1. Trusted cert never generated → run `spark-http-proxy certs generate "*.spark.loc"`. + `spark-http-proxy certs describe ` says whether any installed + certificate covers the hostname, and which one to generate if none does. 2. Warning on a nested domain (`drupal.client.spark.loc`) while `*.spark.loc` exists → wildcards cover one level only; generate the wildcard one level above the host, here `*.client.spark.loc`. See `certificates.md`. diff --git a/skills/system/spark-http-proxy/references/uninstall.md b/skills/system/spark-http-proxy/references/uninstall.md index 0212e30..c23767c 100644 --- a/skills/system/spark-http-proxy/references/uninstall.md +++ b/skills/system/spark-http-proxy/references/uninstall.md @@ -7,7 +7,7 @@ - **Docker** — required; the proxy is a Docker stack. `spark-http-proxy self-test` checks the daemon and reports problems. - **mkcert** — only needed for trusted HTTPS. `command -v mkcert`. You usually do - not check it yourself: `spark-http-proxy generate-mkcert` installs it + not check it yourself: `spark-http-proxy certs generate` installs it automatically (Homebrew on macOS, pacman on Arch) and runs `mkcert -install` to add the local CA. On other Linux distros it cannot auto-install and prints manual steps, so install mkcert + nss first there. `mkcert -CAROOT` shows where