feat(installer,chart): place datasets on a network mount while MySQL stays local

[saadqbal]

What

Storage work for tracebloc/backend#743 — let VMs whose real storage is a network (NFS/CIFS) mount keep the database on local disk while the large dataset volume lives on the network mount.

Stacked on #261 (the preflight guard). Until #261 merges, this PR's diff also includes #261's commit — review/merge #261 first, then this (or review this against fix/743-preflight-network-fs).

Why

MySQL/InnoDB corrupts on NFS and the chart's root chown init-container is blocked by NFS root_squash, so the DB must stay local. But the dataset volume (usually the bulk of storage) can live on the customer's network mount — provided the pod that writes it runs as the uid that owns the export.

Changes

Chart (client/):

• Parameterize the dataset PV hostPath base via hostPath.datasetPath (helper tracebloc.clientDataHostPath), default /tracebloc (byte-identical render). mysql + logs PV paths are untouched. values.yaml + schema + default dict nil-guard for --reuse-values.

Installer (bash + PowerShell):

• New HOST_DATASET_DIR: validated (must exist + be writable; may live outside $HOME, unlike HOST_DATA_DIR; system paths barred), bind-mounted into k3d at a distinct /tracebloc-data path; the dataset dir is created there while mysql + logs stay local.
• When set, the generated values set hostPath.datasetPath=/tracebloc-data and (Linux) pass HOST_UID/HOST_GID env to jobs-manager — consumed by the client-runtime ingestor change (separate PR) so spawned ingestion pods write the host-owned NFS export as the owning uid.
• Preflight notes the dataset dir is exempt from the network-FS hard-fail.

Docs: docs/INSTALL.md checklist + docs/SECURITY.md §5.4.

Tests

• helm-unittest: new shared_images_pvc_test.yaml + mysql/logs split-only guards — 259 pass.
• bats: HOST_DATASET_DIR validation, second-mount, dir-split, values-generation.
• helm lint --strict, shellcheck --severity=error, PowerShell parse — all clean.

Cross-repo

The end-to-end NFS write path also needs the client-runtime ingestor-uid PR (run the ingestion pod as HOST_UID). This PR + that one together complete backend#743's "datasets on NFS".

🤖 Generated with Claude Code

---

Note

Medium Risk
Changes persistent volume paths, k3d bind mounts, and install preflight behavior; misconfiguration could misroute datasets or block installs, but MySQL remains local by design and guards target that failure mode.

Overview
backend#743 splits storage so MySQL and logs stay on local disk while the shared dataset volume can live on a separate network (NFS) mount.

The Helm chart adds hostPath.datasetPath (helper tracebloc.clientDataHostPath) so only the shared-images / dataset PV base path changes; default /tracebloc keeps renders unchanged. MySQL and logs hostPath paths are unchanged.

Installers gain HOST_DATASET_DIR: optional host dir (may be outside $HOME, must exist and be writable), bind-mounted into k3d at /tracebloc-data, with hostPath.datasetPath=/tracebloc-data and (Linux) HOST_UID/HOST_GID in generated values for NFS root_squash ingestion writes. Reusing an existing cluster without that bind mount fails fast so datasets are not placed on ephemeral node storage.

Preflight now hard-fails when HOST_DATA_DIR is on NFS/CIFS/SMB (override TRACEBLOC_ALLOW_NETWORK_FS); HOST_DATASET_DIR is noted as allowed on network FS. Docs (INSTALL.md, SECURITY.md §5.4) and helm/bats/Pester tests cover the split and guards.

^{Reviewed by Cursor Bugbot for commit c31d866. Bugbot is set up for automated code reviews on this repo. Configure here.}

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 096c4bb. Configure here.}

[LukasWodka]

👋 Heads-up — Code review queue is at 18 / 8

Above the WIP limit. The team convention is to review existing PRs before opening new work.

Open PRs currently in Code review (oldest first):

• .github#57 — fix(fr-gate): pass items at or beyond the required stage · author: @aptracebloc · no reviewer assigned
• backend#812 — fix(datasets): subset label check for test-dataset compatibility (#811) · author: @saadqbal · no reviewer assigned
• cli#78 — fix(dataset rm): delete staging files from a uid-65532 pod, not jobs-manager (bug: dataset rm cannot delete staging files — ingestor (uid 65534) vs jobs-manager uid mismatch, no shared fsGroup #259) · author: @LukasWodka · no reviewer assigned
• cli#79 — chore(schema): re-sync vendored ingest.v1.json from data-ingestors master · author: @LukasWodka · no reviewer assigned
• client#261 — feat(installer): fail fast when HOST_DATA_DIR is on a network filesystem · author: @saadqbal · no reviewer assigned
• client-runtime#108 — fix(authz): match ingest table prefixes at a segment boundary (close cross-tenant straddle) · author: @LukasWodka · no reviewer assigned
• client-runtime#114 — fix(jobs): cap training Job backoffLimit to stop crashloops starving the cluster · author: @saadqbal · no reviewer assigned
• client-runtime#115 — feat(ingestion): run the ingestion pod as the host uid for datasets on NFS · author: @saadqbal · no reviewer assigned
• data-ingestors#270 — docs(releasing): correct ingestor rollout — floating tag + imagePullPolicy=Always, not INGESTOR_IMAGE_DIGEST rewrite · author: @saadqbal · no reviewer assigned
• data-ingestors#275 — refactor(P5b): extract the cross-ingest table lock into TableLock · author: @LukasWodka · no reviewer assigned

Pull from review before opening new work. (This is a nudge from the kanban WIP check, not a block.)