From 8b28a43afcf679f464f8040fb0789ee65ad07f81 Mon Sep 17 00:00:00 2001 From: Paolo Di Tommaso Date: Wed, 12 Aug 2026 09:18:35 +0200 Subject: [PATCH] docs: fix blocking errors and trim the self-install guides Follow-up to #1106. Addresses the review findings in three groups. Deployments that would not work as documented: - aws-build: scan was enabled without wave.scan.reports.path. The bean factory throws on a null path and Application starts Micronaut with eagerInitSingletons(true), so Wave failed to boot. - aws-build: wave-build-pvc existed only in the `wave` namespace while build jobs run in `wave-build`. PVCs are namespaced, so every build, scan, and mirror pod stayed Pending. Now created in both. - aws-build: added the missing `noarch` node selector key. Without it getNoArchSelector returns an empty map and mirror and blob cache pods schedule outside the build node group. - aws-build: the S3 bucket referenced by the IAM policy and the build logs and locks paths is now created, and listed as a requirement. - aws-build: restored WAVE_JVM_OPTS. launch.sh defaults to -Xmx850m regardless of the 4Gi container limit. - docker-compose: DOCKER_USER and DOCKER_PAT are not read by Wave. They work only where config.yml interpolates them, and this guide replaces the config.yml that does. Moved into wave.registries. - kubernetes-lite: the registry snippet was a second top-level `wave:` block, which would drop wave.db.* and wave.server.url. Folded into the single ConfigMap. - kubernetes-lite: added the missing `kubectl apply` step, and completed the ALB ingress with ingressClassName and annotations so the /service-info check in post-install can pass. - Restored the endpoints.health guard dropped from master. jdbc and disk-space are on by default, so a brief Postgres blip turned the liveness probe red. - configure-wave: the MICRONAUT_ENVIRONMENTS examples re-added `lite`, which gates out JobManager and cleanup via @Requires(missingBeans). - reference: restored kebab-case on six keys the merge reverted to camelCase, undoing #1096 - they still bind from YAML but lose env-var overridability. - Production checklist now uses wave.capabilities.anonymous-access instead of the wave.allowAnonymous legacy alias, and mentions the strict environment. - Fixed how-wave-works link to the renamed configuration reference. Simplification, per the review rubric: - Moved the seven-registry pre-creation matrix out of the EKS guide and into the reference. - Moved Bottlerocket user-namespace setup to troubleshoot. - Cut the SMTP settings table, the duplicate ECR cache walkthrough, the third IRSA explanation, and the EC2-instance-profile section for Docker builds, which the install paths do not support. - Trimmed the IAM narration to the Wave-specific delta and linked the AWS IRSA procedure. - Dropped "Production enhancements", keeping build-pod egress as a checklist item. - Made the sizing prerequisites per-replica so they match the manifests. Reference and verification: - Added wave.build.enabled, wave.mirror.enabled, wave.blob-cache.enabled, wave.job-manager.max-running-jobs and the mail.smtp.* keys; finished the truncated wave.accounts entry; corrected five (required) labels on settings that ship defaults. - post-install now covers the mirror and scan checks for the features aws-build enables. - Added troubleshoot to the sidebar and gave it front matter; moved post-install ahead of aws-build. Co-Authored-By: Claude Opus 5 (1M context) --- docs/how-wave-works.md | 2 +- docs/install/aws-build.md | 229 +++++++++++++++----------------- docs/install/configure-wave.md | 183 +++---------------------- docs/install/docker-compose.md | 77 ++++++----- docs/install/index.md | 10 +- docs/install/kubernetes-lite.md | 95 +++++++------ docs/install/post-install.md | 16 ++- docs/install/reference.md | 109 ++++++++++++--- docs/sidebar.json | 3 +- docs/troubleshoot.md | 47 ++++++- 10 files changed, 375 insertions(+), 396 deletions(-) diff --git a/docs/how-wave-works.md b/docs/how-wave-works.md index 7f6bde783..72976eba5 100644 --- a/docs/how-wave-works.md +++ b/docs/how-wave-works.md @@ -123,7 +123,7 @@ The lifetime of an ephemeral image that includes an access token in its URI depe - **Other requests.** Builds, mirrors, and requests not bound to a workflow keep a fixed lifetime of 36 hours (the `cache.duration`) from submission. -System administrators can revoke images before these limits. The [`wave.tokens.*`](./configuration.md#general) settings control this behavior. To disable it, set `wave.tokens.watcher.enabled: false`. All requests then use the fixed 36-hour lifetime. +System administrators can revoke images before these limits. The [`wave.tokens.*`](./install/reference.md#general) settings control this behavior. To disable it, set `wave.tokens.watcher.enabled: false`. All requests then use the fixed 36-hour lifetime. ### Context directory size limits diff --git a/docs/install/aws-build.md b/docs/install/aws-build.md index 6d074860d..dfcd350bb 100644 --- a/docs/install/aws-build.md +++ b/docs/install/aws-build.md @@ -8,7 +8,7 @@ The full Wave configuration is a Wave Lite deployment on Amazon EKS with on-dema Build, mirror, and scan are independent toggles. Enable any subset, for example mirror without build. Scan and freeze depend on the build pipeline. With `build.enabled: false`, both are unavailable. :::tip -The Kubernetes manifests in this guide assemble into a single file. Save each YAML block into `wave-build.yaml` in the order shown, separated by `---`, then apply the file once at the end. The AWS CLI steps for ECR and IRSA run on their own and are not part of this file. +The Kubernetes manifests in this guide assemble into a single file. Save each YAML block into `wave-build.yaml` in the order shown, separated by `---`, then apply the file once at the end. The AWS CLI steps (ECR, S3, and IRSA) run on their own and are not part of this file. ::: ## Infrastructure requirements @@ -21,6 +21,7 @@ In addition to the EKS cluster, managed database, and Redis that your Wave Lite | --- | --- | | Amazon Elastic File System (EFS) and its CSI driver | ReadWriteMany build workspace shared across build pods. | | Amazon Elastic Container Registry (ECR) repositories | One for built images, one for the BuildKit layer cache. | +| Amazon S3 bucket | Build logs, build lock files, and scan reports. | | IAM OpenID Connect (OIDC) provider and IAM Roles for Service Accounts (IRSA) role | Wave's AWS identity, used for S3 access and STS role assumption from the cluster. | | Dedicated build node group | Isolates build workloads. Label its nodes `service=wave-build` (and `service=wave-build-arm64` for ARM) to match the node selector. | @@ -29,11 +30,23 @@ In addition to the EKS cluster, managed database, and Redis that your Wave Lite You need the following: - A Wave Lite deployment running on an Amazon EKS cluster (see [Install Wave Lite on Kubernetes](kubernetes-lite.md)). -- Permission to create EFS, ECR, IAM, and node-group resources in the cluster's AWS account. +- Permission to create EFS, ECR, S3, IAM, and node-group resources in the cluster's AWS account. - The AWS CLI and `kubectl`, authenticated against the account. ::: +## Set the shell variables + +The AWS CLI commands in this guide use the following variables. Set them for your environment before you run anything else: + +```bash +export AWS_REGION=us-east-1 +export AWS_ACCOUNT= +export AWS_EKS_CLUSTER_NAME= +export WAVE_S3_BUCKET= +export WAVE_CONFIG_NAME=wave # reused as the IAM policy and role name +``` + ## Create the build namespace Build, scan, and mirror pods run in a dedicated namespace: @@ -56,37 +69,17 @@ aws --region "$AWS_REGION" ecr create-repository --repository-name wave/build aws --region "$AWS_REGION" ecr create-repository --repository-name wave/cache ``` -ECR requires repositories to exist before push. Registries differ in whether they need the target path to exist beforehand. Wave relies on the underlying push tool: BuildKit for builds, Skopeo for mirrors. If you point builds or mirroring at a registry that requires pre-creation and the target path is missing, the push fails partway through the layer upload. Check your target registry's rules before you push. - -For credential syntax, see [Configure Wave](configure-wave.md). For common push and authentication failures, see [Registry push and authentication failures](../troubleshoot.md#registry-push-and-authentication-failures). - -### AWS ECR - -ECR **requires pre-creation**. Each repository must exist before push. Create it with `aws ecr create-repository` (as shown earlier) or your IaC tooling. AWS offers registry-level auto-create policies, but they are not enabled by default. +ECR does not auto-create repositories, so both must exist before Wave's first push. If you point builds or mirroring at a registry other than ECR, check its rules first: see [Registry pre-creation](reference.md#registry-pre-creation) for the per-registry matrix, and [Registry push and authentication failures](../troubleshoot.md#registry-push-and-authentication-failures) when a push fails partway through. -### Docker Hub +## Create the S3 bucket -Docker Hub **needs no pre-creation**. Repositories auto-create in your user or organization namespace on first push. Repository-count and pull rate limits apply on paid tiers. +Wave writes build logs, build lock files, and scan reports to S3: -### GitHub Container Registry (GHCR) - -GHCR **needs no pre-creation**. Repositories auto-create under your user or organization namespace. Default visibility inherits from the organization's package settings unless you override it. - -### Google Artifact Registry - -Google Artifact Registry **requires partial pre-creation**. The repository, which is the GCP namespace container, must be pre-created with `gcloud artifacts repositories create`. Image paths within it auto-create on push. - -### Google Container Registry (legacy GCR) - -Legacy GCR **needs no pre-creation**. Repositories auto-create on push. Google is phasing out GCR. Target Artifact Registry for new deployments. - -### Azure Container Registry (ACR) - -ACR **needs no pre-creation** for image paths. Image paths auto-create within an existing ACR instance. The instance must exist first. Wave must also hold the `AcrPush` role on it. - -### Harbor +```bash +aws --region "$AWS_REGION" s3 mb "s3://$WAVE_S3_BUCKET" +``` -Harbor **requires partial pre-creation**. Projects must be pre-created through the UI or API. Images within a project auto-create if project policies permit. +For naming rules and bucket options, see [Creating a bucket](https://docs.aws.amazon.com/AmazonS3/latest/userguide/create-bucket-overview.html). ## Grant Wave access to AWS APIs with IRSA @@ -95,29 +88,16 @@ IRSA gives the Wave pod its AWS identity. Wave uses it directly for S3 (build lo - An IAM access key pair. Attach the ECR statements of the policy in this section to that IAM user. - The ARN of an IAM role whose trust policy allows the Wave role to call `sts:AssumeRole`. Attach the ECR statements to that role. -IRSA requires an IAM OIDC provider for the cluster. To check whether one exists or create it, see [Creating an IAM OIDC provider](https://docs.aws.amazon.com/eks/latest/userguide/enable-iam-roles-for-service-accounts.html). - -The commands in this section use the following variables. Set them for your environment first: - -```bash -export AWS_REGION=us-east-1 -export AWS_ACCOUNT= -export AWS_EKS_CLUSTER_NAME= -export WAVE_CONFIG_NAME=wave # reused as the IAM policy and role name -``` +IRSA requires an IAM OIDC provider for the cluster, and the mechanics of associating a role with a service account are AWS's, not Wave's: see [Assign IAM roles to Kubernetes service accounts](https://docs.aws.amazon.com/eks/latest/userguide/associate-service-account-role.html). What follows is the Wave-specific part — which permissions to grant, and which service account to bind them to. -Find the cluster's OIDC issuer URL: +Author the two policy documents shown below, then create and attach the role. The `describe-cluster` call prints the OIDC issuer URL that `seqera-wave-role.json` needs: ```bash aws --region "$AWS_REGION" eks describe-cluster \ --name "$AWS_EKS_CLUSTER_NAME" \ --query "cluster.identity.oidc.issuer" \ --output text -``` - -Create an IAM policy and role, then attach them. The two documents (`seqera-wave-policy.json` and `seqera-wave-role.json`) are templates you author for your account, shown after the commands: -```bash aws --region "$AWS_REGION" iam create-policy \ --policy-name "$WAVE_CONFIG_NAME" \ --policy-document file://seqera-wave-policy.json @@ -131,7 +111,7 @@ aws --region "$AWS_REGION" iam attach-role-policy \ --policy-arn "arn:aws:iam::$AWS_ACCOUNT:policy/$WAVE_CONFIG_NAME" ``` -`seqera-wave-policy.json` grants access to ECR (for built and cached images) and S3 (for build logs and Conda lock files). Scope each resource to your repository and bucket ARNs. Replace `` and `` with the `$AWS_REGION` and `$AWS_ACCOUNT` values you exported earlier: +`seqera-wave-policy.json` grants access to ECR (for built and cached images) and S3 (for build logs, lock files, and scan reports). Replace ``, ``, and `` with the values you exported earlier: ```json { @@ -172,7 +152,7 @@ aws --region "$AWS_REGION" iam attach-role-policy \ "Resource": ["arn:aws:ecr:::repository/wave/*"] }, { - "Sid": "BuildLogsAndLocks", + "Sid": "BuildLogsLocksAndScanReports", "Effect": "Allow", "Action": ["s3:GetObject", "s3:PutObject", "s3:ListBucket"], "Resource": [ @@ -219,9 +199,12 @@ metadata: eks.amazonaws.com/role-arn: arn:aws:iam:::role/ ``` -This single IAM role is the identity the Wave service pod uses for its AWS API calls (S3, and STS when assuming a role for ECR token exchange) through the `wave-sa` service account. Build, scan, and mirror pods run as `wave-build-sa` and do not inherit this identity. They authenticate to registries with configuration files that Wave writes to the shared build workspace. If IRSA is unavailable, attach an EC2 instance profile carrying the same policy to the node group that runs the Wave service pod instead. +Only the Wave service pod uses this role, through the `wave-sa` service account. Build, scan, and mirror pods run as `wave-build-sa`, which needs no AWS identity — Wave writes registry credentials into the shared build workspace for them. If IRSA is unavailable, attach an EC2 instance profile carrying the same policy to the node group that runs the Wave service pod. -Wave routes each request to one credential source. Requests that carry a Platform identity use the workspace credentials the user configures. The exception is operator-owned targets (the registry hosts of `wave.build.repo`, `wave.build.cache`, and `wave.build.public-repo`), which always use the server-side static credentials (`wave.registries..username` and `.password`). Anonymous requests always use the server-side credentials. The cloud identity alone does not authenticate to registries. For ECR, Wave exchanges the configured `wave.registries` credentials (an access key pair, or a role ARN it assumes via STS) for an ECR auth token. +Two rules matter here, on top of the [credential sources](kubernetes-lite.md#registry-credentials) Wave Lite already uses: + +- Operator-owned targets — the registry hosts of `wave.build.repo`, `wave.build.cache`, and `wave.build.public-repo` — always use the server-side `wave.registries.` credentials, never a user's workspace credentials. +- The cloud identity does not authenticate to registries by itself. For ECR, Wave exchanges the configured `wave.registries` credentials (an access key pair, or a role ARN it assumes via STS) for an ECR auth token. ## Configure EFS storage @@ -243,9 +226,9 @@ parameters: directoryPerms: "0755" ``` -### Persistent volume +### Persistent volumes and claims -Create a persistent volume bound to your EFS file system: +The Wave service pod and the build pods each mount the workspace, and they run in different namespaces. Persistent volume claims are namespaced, so the claim must exist in **both** `wave` and `wave-build` — a claim only in `wave` leaves every build, scan, and mirror pod stuck in `Pending`. Create one volume and one claim per namespace, all pointing at the same EFS file system: ```yaml apiVersion: v1 @@ -260,16 +243,13 @@ spec: - ReadWriteMany persistentVolumeReclaimPolicy: Retain storageClassName: efs-wave-sc + claimRef: + namespace: wave + name: wave-build-pvc csi: driver: efs.csi.aws.com volumeHandle: "" -``` - -### Persistent volume claim - -Claim the volume for the build workspace: - -```yaml +--- apiVersion: v1 kind: PersistentVolumeClaim metadata: @@ -284,11 +264,47 @@ spec: requests: storage: 500Gi storageClassName: efs-wave-sc + volumeName: wave-build-pv +--- +apiVersion: v1 +kind: PersistentVolume +metadata: + name: wave-build-pv-jobs +spec: + capacity: + storage: 500Gi + volumeMode: Filesystem + accessModes: + - ReadWriteMany + persistentVolumeReclaimPolicy: Retain + storageClassName: efs-wave-sc + claimRef: + namespace: wave-build + name: wave-build-pvc + csi: + driver: efs.csi.aws.com + volumeHandle: "" +--- +apiVersion: v1 +kind: PersistentVolumeClaim +metadata: + namespace: wave-build + name: wave-build-pvc + labels: + app: wave-app +spec: + accessModes: + - ReadWriteMany + resources: + requests: + storage: 500Gi + storageClassName: efs-wave-sc + volumeName: wave-build-pv-jobs ``` Configuration notes: -- Replace `` with your EFS file system ID. +- Replace `` with your EFS file system ID in both volumes. Both claims keep the name `wave-build-pvc`, because `wave.build.k8s.storage.claim-name` is a single value used in both namespaces. - The EFS security group must allow inbound and outbound NFS traffic (port `2049`) from the EKS worker nodes. ## Create the build RBAC @@ -362,12 +378,15 @@ data: namespace: "wave-build" service-account: "wave-build-sa" storage: - claimName: "wave-build-pvc" - mountPath: "/efs/wave/build" + claim-name: "wave-build-pvc" + mount-path: "/efs/wave/build" node-selector: # Keys are container platforms. Values are 'label=value' applied to your build node groups. + # 'noarch' is required: mirror and blob cache pods are architecture-independent and + # get no node selector at all without it. 'linux/amd64': 'service=wave-build' 'linux/arm64': 'service=wave-build-arm64' + 'noarch': 'service=wave-build' logs: path: "s3:///wave/build-logs" locks: @@ -377,11 +396,12 @@ data: enabled: true scan: enabled: true + # Required whenever scan is enabled: Wave fails to start without it. + reports: + path: "s3:///wave/scan-reports" blobCache: enabled: false # Enabling blob cache needs S3. See Configure Wave. - # ECR registry credentials. Wave exchanges these for ECR auth tokens. - # Use an IAM access key pair, or an IAM role ARN as username (with an - # optional external ID as password) that Wave assumes via STS using + # ECR registry credentials. Wave exchanges these for ECR auth tokens using # its IRSA identity. See "Grant Wave access to AWS APIs with IRSA". registries: .dkr.ecr..amazonaws.com: @@ -399,16 +419,22 @@ data: tower: endpoint: url: "https://platform.example.com/api" + endpoints: + health: + enabled: true + disk-space: + enabled: false + jdbc: + enabled: false ``` -Wave ships working defaults for the build tool images and timeout. You do not need to set them. The defaults are: +Wave ships working defaults for the build tool images and the build timeout, so the ConfigMap does not set them. To override one, or to tune the build subsystem further, see [Container build process](reference.md#container-build-process). -- `wave.build.buildkit-image`: `public.cr.seqera.io/wave/buildkit:v0.25.2-rootless` -- `wave.build.singularity-image`: `public.cr.seqera.io/wave/singularity:v4.2.1-r4` -- `wave.blobCache.s5cmdImage`: `public.cr.seqera.io/wave/s5cmd:v2.3.0` -- `wave.build.timeout`: `900s` (15 minutes) +To build ARM (Graviton) images, route `linux/arm64` builds to an ARM node group with the `node-selector` shown earlier. For cache setup (ECR cache repository, S3 cache authentication), see [Configure Wave](configure-wave.md). -To build ARM (Graviton) images, route `linux/arm64` builds to an ARM node group with the `node-selector` shown earlier. For deeper build and blob cache tuning, see [Container build process](reference.md#container-build-process). For cache setup (ECR cache repository, S3 cache authentication), see [Configure Wave](configure-wave.md). +:::note +If your build nodes run Bottlerocket, BuildKit needs user namespaces enabled before any build succeeds. See [Builds fail on Bottlerocket nodes](../troubleshoot.md#builds-fail-on-bottlerocket-nodes). +::: ## Update the Wave deployment @@ -445,6 +471,17 @@ spec: - name: MICRONAUT_ENVIRONMENTS # lite is dropped from the Lite set; k8s enables the in-cluster build client. value: "postgres,redis,k8s,rate-limit" + # The image defaults to an 850 MB heap regardless of the container limit. + # Setting this variable replaces the whole default option set, so keep the + # flags below alongside your own. + - name: WAVE_JVM_OPTS + value: >- + -XX:+UseG1GC + -Xms1g + -Xmx3g + -XX:MaxDirectMemorySize=100m + -Dio.netty.maxDirectMemory=0 + -Dio.netty.allocator.type=pooled resources: requests: memory: "4Gi" @@ -492,58 +529,8 @@ kubectl logs -f deployment/wave -n wave | grep -i build In freeze mode, a pipeline sets `wave.build.repository` (the Nextflow-side setting) to choose its own push target. Wave treats the value as custom only if it sits outside the operator's `wave.build.repo`, `wave.build.public-repo`, and `wave.build.cache` prefixes. If it starts with one of those prefixes, Wave rejects the freeze with a `must be specified when using freeze mode` error (with a numbered suffix such as `[1]`), even though the pipeline did supply a value. -To let users freeze to their own repositories, reserve a registry namespace outside your operator prefixes and distribute push credentials through Platform workspaces. See [Create the ECR repositories](#create-the-ecr-repositories) for per-registry pre-creation rules. - -## Bottlerocket support - -BuildKit requires user namespaces, but Bottlerocket sets `user.max_user_namespaces=0` by default. Enable user namespaces on your build nodes by setting `user.max_user_namespaces` to a sufficiently high value (for example, `63359`, as in the DaemonSet that follows). Values that are too low limit concurrent build capacity and can cause build failures. - -Set this at boot through your node group's startup script or user data (preferred, with no privileged containers required). If you cannot control node configuration directly, apply it with a DaemonSet that runs a privileged container on build nodes only: - -```yaml -apiVersion: apps/v1 -kind: DaemonSet -metadata: - labels: - app: sysctl-userns - name: sysctl-userns -spec: - selector: - matchLabels: - app: sysctl-userns - template: - metadata: - labels: - app: sysctl-userns - spec: - containers: - - name: sysctl-userns - image: busybox - command: ["sh", "-euxc", "sysctl -w user.max_user_namespaces=63359 && sleep infinity"] - securityContext: - privileged: true - affinity: - nodeAffinity: - requiredDuringSchedulingIgnoredDuringExecution: - nodeSelectorTerms: - - matchExpressions: - - key: service - operator: In - values: ["wave-build", "wave-build-arm64"] -``` - -For more on Bottlerocket, see the [Bottlerocket FAQs](https://bottlerocket.dev/en/faq/). - -## Production enhancements - -For production build deployments, consider: - -- **Dedicated build node pools** to isolate build workloads. -- **A ResourceQuota** on the build namespace, paired with `wave.job-manager.max-running-jobs`, to cap concurrent build resource usage. -- **NetworkPolicies** restricting build-pod egress to registries and S3 only. Build pods run user-submitted Dockerfiles. -- **EFS access points** to isolate build workspaces. -- **Monitoring** of build success/failure rates, duration, EFS usage, and queue length. +To let users freeze to their own repositories, reserve a registry namespace outside your operator prefixes and distribute push credentials through Platform workspaces. See [Registry pre-creation](reference.md#registry-pre-creation) for whether the target registry needs the repository to exist first. ## Verify your installation -Run the build functional checks in [Verify your installation](post-install.md), then continue to the [production checklist](configure-wave.md#production-checklist) to prepare the deployment for production. +Run the build, mirror, and scan functional checks in [Verify your installation](post-install.md), then continue to the [production checklist](configure-wave.md#production-checklist) to prepare the deployment for production. diff --git a/docs/install/configure-wave.md b/docs/install/configure-wave.md index e98cbe74b..0e3230380 100644 --- a/docs/install/configure-wave.md +++ b/docs/install/configure-wave.md @@ -15,15 +15,18 @@ A freshly installed Wave service boots and returns `200` on `/service-info`, but ### Require authentication -By default Wave allows anonymous pulls (`wave.allowAnonymous: true`). In production, require authentication so only paired Platform clients can request containers: +By default Wave allows anonymous pulls. In production, require authentication so only paired Platform clients can request containers: ```yaml wave: - allowAnonymous: false + capabilities: + anonymous-access: false ``` With anonymous access disabled, every request must carry a valid Platform-issued token. +For a regulated deployment that must also stop Wave from serving images directly, brokering registry credentials, or exposing its HTML pages, add `strict` to `MICRONAUT_ENVIRONMENTS` instead. That disables all four capability toggles at once. See [Capabilities](reference.md#capabilities). + ### Terminate TLS Wave does not terminate TLS itself. Front it with an ingress or load balancer that holds the certificate. For example, an Application Load Balancer (ALB) with an AWS Certificate Manager (ACM) certificate matching the Wave hostname, and a Route 53 alias record pointing at the load balancer. Confirm `wave.server.url` uses the `https://` hostname clients reach. @@ -59,8 +62,14 @@ Builds and augmented images accumulate. Set cleanup and retention so storage sta Reserve about 2 GB memory and 0.2 CPU per Wave instance, with limits of 4 GB and 1 CPU, matching the sizing in the install paths. Run multiple replicas behind the load balancer for availability. +Set `WAVE_JVM_OPTS` to match the container limit. The image defaults to an 850 MB heap whatever the limit says, so a 4 GB container leaves most of its memory unused until you override it. Setting the variable replaces the whole default option set rather than adding to it, so copy the defaults from `src/main/jib/launch.sh` and adjust `-Xmx`. + Size the build node pool and cap concurrency with `wave.job-manager.max-running-jobs` and a build-namespace `ResourceQuota`. +### Restrict build-pod egress + +Build pods run user-supplied Dockerfiles. On a build-enabled deployment, apply a `NetworkPolicy` to the build namespace that limits egress to your registries and S3. + ### Review security headers Wave sends HTTP security headers (HSTS, frame options, content-type options, referrer policy, permissions policy, and a content security policy) by default. Review them against your environment and adjust the content security policy if you front Wave with additional origins. See [Security headers](reference.md#security-headers) in the Configuration reference. @@ -71,14 +80,7 @@ Wave sends email notifications for build-related events. Configure delivery thro ### SMTP -Add `mail` to your Micronaut environments: - -```yaml -# Add 'mail' to your existing environments -MICRONAUT_ENVIRONMENTS: "lite,postgres,redis,rate-limit,mail" -``` - -Configure the SMTP settings in your Wave configuration: +Append `mail` to the `MICRONAUT_ENVIRONMENTS` value your install path already sets, then configure the SMTP settings in your Wave configuration: ```yaml mail: @@ -96,17 +98,7 @@ mail: protocols: "TLSv1.2" ``` -| Setting | Description | Example values | -| ------------------- | ------------------------------------ | ------------------------------------------- | -| `from` | Email address that appears as sender | `wave@company.com` | -| `host` | SMTP server hostname | `smtp.gmail.com`, `smtp.office365.com` | -| `port` | SMTP server port | `587` (STARTTLS), `465` (SSL), `25` (plain) | -| `user` | SMTP authentication username | Usually your email address | -| `password` | SMTP authentication password | App password or account password | -| `auth` | Enable SMTP authentication | `true` (recommended) | -| `starttls.enable` | Enable STARTTLS encryption | `true` (recommended) | -| `starttls.required` | Require STARTTLS encryption | `true` (recommended) | -| `ssl.protocols` | Supported SSL/TLS protocols | `TLSv1.2`, `TLSv1.3` | +For every `mail.*` setting, see [Email configuration](reference.md#email-configuration). ### SES @@ -123,14 +115,7 @@ You need the following: ::: -Add `aws-ses` to your Micronaut environments along with `mail`: - -```yaml -# Add both 'mail' and 'aws-ses' to your existing environments -MICRONAUT_ENVIRONMENTS: "lite,postgres,redis,rate-limit,mail,aws-ses" -``` - -Set the sender address in your Wave configuration: +Append both `mail` and `aws-ses` to the `MICRONAUT_ENVIRONMENTS` value your install path already sets, then set the sender address in your Wave configuration: ```yaml mail: @@ -182,151 +167,27 @@ wave: Wave runs scans with its bundled Trivy-based scanner image. Override the image with `wave.scan.image.name` if you mirror it to your own registry. For all scan options, see [Container scan process](reference.md#container-scan-process). -## ECR cache repository - -Use Amazon Elastic Container Registry (ECR) as a cache repository to store and reuse build layers. Reusing cached layers speeds up builds and reduces bandwidth. ECR cache requires the Wave build service and works only in AWS deployments with ECR access. - -:::info[**Prerequisites**] - -You need the following: - -- An ECR repository in the same AWS region as Wave. -- IAM permissions for Wave to push to and pull from ECR. -- An ECR repository reachable from the Wave build infrastructure. - -::: - -Configure the ECR cache repository in your Wave configuration: - -```yaml -wave: - build: - enabled: true - cache: ".dkr.ecr..amazonaws.com/wave-cache" -``` - -[Enable Wave builds](aws-build.md) defines the ECR cache IAM permissions. Add your cache ARN as an allowed `Resource`. - -To create and configure the ECR cache repository: - -1. Create the ECR repository: - - ```bash - aws ecr create-repository --repository-name wave-cache --region us-east-1 - ``` +## Build layer cache -2. Configure a lifecycle policy to manage cache storage costs: - ```json - { - "rules": [ - { - "rulePriority": 1, - "selection": { - "tagStatus": "untagged", - "countType": "sinceImagePushed", - "countUnit": "days", - "countNumber": 7 - }, - "action": { - "type": "expire" - } - } - ] - } - ``` +`wave.build.cache` takes either a container repository or an S3 path. [Enable Wave builds](aws-build.md) sets it to the ECR repository created there, which is the default choice on AWS; add a [lifecycle policy](https://docs.aws.amazon.com/AmazonECR/latest/userguide/LifecyclePolicies.html) expiring untagged images to keep its storage bounded. -The `wave.build.cache` setting takes a cache repository URL or an S3 path. For details, see [Container build process](reference.md#container-build-process). - -## S3 cache authentication - -When you set `wave.build.cache` to an S3 bucket path, Wave uses S3 as the BuildKit cache backend. Wave authenticates with native AWS mechanisms instead of static credentials in configuration files. - -For the related configuration options (`wave.build.cache`, `wave.build.cache-bucket-region`, `wave.build.cache-bucket-upload-parallelism`), see [Container build process](reference.md#container-build-process). - -### Kubernetes deployments - -S3 cache uses IAM Roles for Service Accounts (IRSA) for credential-free authentication. - -Configure your Kubernetes ServiceAccount with an IAM role annotation: - -```yaml -apiVersion: v1 -kind: ServiceAccount -metadata: - name: wave-build-sa - namespace: wave-build - annotations: - eks.amazonaws.com/role-arn: arn:aws:iam::ACCOUNT_ID:role/WaveBuildRole -``` - -The IAM role must have permissions to access the S3 cache bucket: - -```json -{ - "Version": "2012-10-17", - "Statement": [ - { - "Effect": "Allow", - "Action": [ - "s3:PutObject", - "s3:GetObject", - "s3:DeleteObject", - "s3:ListBucket", - "s3:AbortMultipartUpload", - "s3:ListMultipartUploadParts", - "s3:ListBucketMultipartUploads" - ], - "Resource": [ - "arn:aws:s3:::my-bucket/wave/cache", - "arn:aws:s3:::my-bucket/wave/cache/*" - ] - } - ] -} -``` - -No change to the Wave deployment is needed. Build pods already run as `wave-build-sa` through the `wave.build.k8s.service-account` setting described in [Enable Wave builds](aws-build.md#enable-build-features). Keep the Wave deployment on its `wave-sa` service account, which holds the IRSA identity for build logs and lock files. - -### Docker deployments - -For Docker-based builds, use an EC2 instance profile for automatic credential management. - -Attach an IAM role with the S3 permissions shown earlier to the EC2 instance running Docker. BuildKit uses the instance metadata service to obtain temporary credentials. The AWS SDK in BuildKit discovers and uses the instance profile credentials. No further configuration is required. - -:::note -For development and testing only, you can provide AWS credentials through environment variables: - -```bash -export AWS_ACCESS_KEY_ID=your_access_key -export AWS_SECRET_ACCESS_KEY=your_secret_key -export AWS_REGION=us-east-1 -``` - -**Warning:** Do not use this approach in production. It requires managing static credentials. Use an EC2 instance profile for production Docker deployments. -::: - -### Configuration example +To use S3 as the BuildKit cache backend instead, point `wave.build.cache` at a bucket path: ```yaml wave: build: cache: "s3://wave-cache-bucket/buildkit" - cache-bucket-region: "us-east-1" # Optional if AWS_REGION is set - cache-bucket-upload-parallelism: 8 # Optional, controls parallel S3 uploads + cache-bucket-region: "us-east-1" # Optional if AWS_REGION is set + cache-bucket-upload-parallelism: 8 # Optional, controls parallel S3 uploads ``` +S3 cache needs no static credentials. Build pods pick up the AWS identity of their node or service account, so extend the IRSA policy from [Enable Wave builds](aws-build.md#grant-wave-access-to-aws-apis-with-irsa) with `s3:PutObject`, `s3:GetObject`, `s3:DeleteObject`, `s3:ListBucket`, `s3:AbortMultipartUpload`, `s3:ListMultipartUploadParts`, and `s3:ListBucketMultipartUploads` on the cache path. For the full set of build cache settings, see [Container build process](reference.md#container-build-process). + ## Client IP address resolution Wave uses client IP addresses for rate limiting. By default, Wave reads the socket address, which clients cannot spoof. -For AWS ALB deployments, add `alb` to your Micronaut environments: - -```yaml -# Add 'alb' to your existing environments -MICRONAUT_ENVIRONMENTS: "lite,postgres,redis,rate-limit,alb" -``` - -The `alb` profile trusts the `X-Forwarded-For` header from the ALB to resolve the client IP. +For AWS ALB deployments, append `alb` to the `MICRONAUT_ENVIRONMENTS` value your install path already sets. The `alb` profile trusts the `X-Forwarded-For` header from the ALB to resolve the client IP. :::warning Enable the `alb` profile only when Wave runs behind a trusted ALB. If Wave is exposed directly to the internet, trusting `X-Forwarded-For` lets clients spoof their IP address and bypass rate limiting. diff --git a/docs/install/docker-compose.md b/docs/install/docker-compose.md index ee59c478b..0606d569f 100644 --- a/docs/install/docker-compose.md +++ b/docs/install/docker-compose.md @@ -5,21 +5,15 @@ description: Deploy Wave Lite on a single Docker host with external PostgreSQL a Install Wave Lite with Docker Compose when you want the Lite configuration without Kubernetes, for example a compliance-constrained site that cannot run EKS. This installs container augmentation, inspection, and private registry authentication. A Docker Compose deployment cannot be extended to the full Wave configuration, which requires Kubernetes on Amazon EKS. -For other choices (embedded databases, or exposing Wave behind HTTPS), see [Adapt this guide](#adapt-this-guide). - :::info[**Prerequisites**] You need the following: - Current, supported versions of Docker Engine and Docker Compose. -- A host that meets the Wave service's minimum compute requirements: - - Memory: 12 GB RAM (4 GB for each of two Wave replicas, plus headroom for the OS and Docker). - - CPU: 4 cores (1 core for each of two Wave replicas, plus headroom for the OS and Docker). - - Storage: 10 GB, plus disk space for your container images and temporary files. - - Network: Connectivity to your PostgreSQL and Redis instances. - - On AWS EC2, an `m5a.2xlarge` instance. -- PostgreSQL 16 or later. -- Redis 6.2 or later. +- A host with capacity for each Wave replica you run. One replica reserves 2 GB RAM and 0.2 CPU and is limited to 4 GB and 1 CPU, so budget 6 GB and 2 cores per replica including headroom for the OS and Docker. On AWS EC2, an `m5a.xlarge` runs one replica comfortably. +- 10 GB storage, plus disk space for container images and temporary files. +- PostgreSQL 16 or later, reachable from the host. +- Redis 6.2 or later, reachable from the host. - A Seqera Platform deployment and its endpoint URL. - Access to the Wave container image from `cr.seqera.io`, using credentials provided by Seqera. @@ -83,7 +77,7 @@ Set `WAVE_SERVER_URL` to the address your clients use to reach Wave. If you leav ## Configure Wave -Create `config.yml` alongside `wave.env`. Wave Lite runs with build, mirror, scan, and blob cache disabled: +Wave requires a `config.yml` in its working directory and fails to start without one. Create it alongside `wave.env`: ```yaml wave: @@ -96,42 +90,49 @@ wave: enabled: false blobCache: enabled: false + +# Keep the JDBC and disk-space indicators out of /health. Micronaut enables +# them by default, so a brief database blip turns the healthcheck below red +# and, under Swarm, gets the task replaced. +endpoints: + health: + enabled: true + disk-space: + enabled: false + jdbc: + enabled: false ``` -This file sets only what Wave Lite needs to start. The `lite` entry in `MICRONAUT_ENVIRONMENTS`, set in the Compose file in a later step, already applies these same defaults. The file restates them explicitly and gives you a place to add further configuration. To configure other options, such as rate limits, token cache duration, and metrics, see [Configure Wave](configure-wave.md). Before serving production traffic, complete the [production checklist](configure-wave.md#production-checklist). +The `lite` entry in `MICRONAUT_ENVIRONMENTS`, set in the Compose file in a later step, already applies the four feature toggles. The file restates them explicitly and gives you a place to add further configuration. For every available setting, see the [Configuration reference](reference.md). Before serving production traffic, complete the [production checklist](configure-wave.md#production-checklist). ## Authenticate to private registries Wave Lite pulls images during augmentation. To augment images from a private registry, give Wave credentials for that registry. Wave uses one of two credential sources per request: - **Platform workspace credentials**: credentials a user adds to their Seqera Platform workspace. Wave uses these for requests that carry a Platform identity. -- **Server-side static credentials**: credentials the operator sets. Wave uses these for anonymous requests and for registries the operator owns. - -For the common registries, set the credentials as environment variables in `wave.env`: - -```bash -# Docker Hub -DOCKER_USER= -DOCKER_PAT= -``` - -```bash -# Quay.io -QUAY_USER= -QUAY_PAT= -``` +- **Server-side static credentials**: credentials the operator sets under `wave.registries.`. Wave uses these for anonymous requests and for registries the operator owns. -For any other registry, add an entry under `wave.registries.` in `config.yml`: +Add an entry per registry to `config.yml`. Wave reads static credentials only from `wave.registries`, so keep the values out of `wave.env` unless you interpolate them here: ```yaml wave: registries: + docker.io: + username: "${DOCKER_USER:}" + password: "${DOCKER_PAT:}" + quay.io: + username: "${QUAY_USER:}" + password: "${QUAY_PAT:}" myregistry.example.com: username: "" password: "" ``` -Configure credentials for every private registry Wave pulls from. Public images need none. For all registry options, see [Container registry](reference.md#container-registry). +The `${VAR:}` form reads the value from the environment, so with the block above you can put `DOCKER_USER` and `DOCKER_PAT` in `wave.env` and keep the secrets out of `config.yml`. Configure credentials for every private registry Wave pulls from. Public images need none. For all registry options, see [Container registry](reference.md#container-registry). + +:::warning +Anonymous access is enabled by default, so any client that can reach Wave can use these operator credentials to pull through it. Disable it with `wave.capabilities.anonymous-access: false` before you expose the service — see [Require authentication](configure-wave.md#require-authentication). +::: ## Log in to the Seqera container registry @@ -180,13 +181,16 @@ services: ## Start Wave -Docker Compose runs Wave in one of two modes, depending on whether you need more than one replica: +Start the service: -- **Single host**: Run `docker compose up -d`. This starts one Wave replica on the local Docker host. -- **Swarm (two or more replicas)**: Initialize Swarm with `docker swarm init` if the host is not already a Swarm manager. Set `replicas: 2` in `docker-compose.yml`, then run `docker stack deploy -c docker-compose.yml wave`. Swarm ignores the `restart` key and applies its default restart policy. Set `deploy.restart_policy` to change it. +```bash +docker compose up -d +``` On first startup, Wave takes 30 to 60 seconds to initialize while it applies database migrations. +For two or more replicas, raise `replicas` and deploy the same file as a Swarm stack instead — see [Deploy a stack to a swarm](https://docs.docker.com/engine/swarm/stack-deploy/). + :::warning If Wave Lite runs in the same Swarm as Platform Connect for [Studios](https://docs.seqera.io/platform-enterprise/25.2/enterprise/studios#docker-compose), removing the stack also interrupts Connect services. ::: @@ -195,11 +199,4 @@ If Wave Lite runs in the same Swarm as Platform Connect for [Studios](https://do Confirm the service is live and functional. See [Verify your installation](post-install.md) for the `/service-info` check and the Wave CLI functional checks. -When Wave is running and verified, continue to the [production checklist](configure-wave.md#production-checklist) to prepare the deployment for production. - -## Adapt this guide - -The supported procedure uses managed PostgreSQL and Redis and assumes you front Wave yourself. The following options are described but not part of the procedure. Adapt them at your own risk. - -- **Embedded PostgreSQL and Redis**: Suitable for development and testing. Not supported for production. -- **Expose Wave externally over HTTPS**: Front the service with a load balancer and certificate (for example, AWS ALB with ACM and Route 53). +When Wave is running and verified, continue to the [production checklist](configure-wave.md#production-checklist) to prepare the deployment for production. That is also where TLS termination is covered: this procedure assumes managed PostgreSQL and Redis, and that you front Wave with your own load balancer. diff --git a/docs/install/index.md b/docs/install/index.md index 74a83549d..26a646774 100644 --- a/docs/install/index.md +++ b/docs/install/index.md @@ -10,24 +10,22 @@ You can self-host Wave in two configurations: - **Wave Lite**: Container augmentation, inspection, and private registry authentication. - **Wave**: Everything in Wave Lite, plus on-demand container builds, freeze, mirroring, and security scanning. -Wave builds on Wave Lite. The full configuration is a Wave Lite deployment with build, mirror, and scan added. To run it, install Wave Lite on an Amazon EKS cluster, then enable those features. +Every deployment starts with Wave Lite. The full configuration is a Wave Lite deployment on Amazon EKS with build, mirror, and scan enabled on top. For the full capability comparison, see the [feature matrix](../features/index.mdx). For how Wave provisions containers, see [How Wave works](../how-wave-works.md). ## Install path -Every deployment starts with Wave Lite. Choose the path that matches your infrastructure and whether you need the full Wave configuration. +Choose the path that matches your infrastructure. ### Docker Compose -Run Wave Lite on a single Docker host without Kubernetes. Choose this path for a compliance-constrained site that cannot run Amazon EKS, or a deployment too small to need a cluster. Docker Compose runs Wave Lite only. The full configuration requires EKS. +Run Wave Lite on a single Docker host without Kubernetes. Choose this path for a compliance-constrained site that cannot run Amazon EKS, or a deployment too small to need a cluster. [Install Wave Lite with Docker Compose](docker-compose.md) ### Kubernetes -Run Wave Lite on a Kubernetes cluster you already operate. This is also the path to the full Wave configuration. Install Wave Lite on Amazon EKS, then [enable Wave builds](aws-build.md) to add on-demand builds, freeze, mirroring, and scanning. Wave Lite has no AWS dependency and runs on any conformant Kubernetes distribution. Only the full configuration requires EKS. - -You can deploy with raw manifests, or with the official [Seqera Wave Helm chart](https://artifacthub.io/packages/helm/seqera/wave) alongside the deployment notes in the Kubernetes guide. +Run Wave Lite on a Kubernetes cluster you already operate — it has no AWS dependency and runs on any conformant distribution. This is also the path to the full Wave configuration: install Wave Lite on Amazon EKS, then [enable Wave builds](aws-build.md) to add on-demand builds, freeze, mirroring, and scanning. [Install Wave Lite on Kubernetes](kubernetes-lite.md) diff --git a/docs/install/kubernetes-lite.md b/docs/install/kubernetes-lite.md index 400799463..ea4e1bb60 100644 --- a/docs/install/kubernetes-lite.md +++ b/docs/install/kubernetes-lite.md @@ -5,24 +5,23 @@ description: Deploy Wave Lite on any Kubernetes cluster with external PostgreSQL Install Wave Lite on a Kubernetes cluster you already operate. This installs container augmentation, inspection, and private registry authentication. Build, mirror, and scan are not part of Wave Lite. The full Wave configuration adds them to a Wave Lite install on Amazon EKS. After you install Wave Lite on an EKS cluster, follow [Enable Wave builds](aws-build.md). -For other choices (different ingress controllers or untested distributions), see [Adapt this guide](#adapt-this-guide). - :::info[**Prerequisites**] You need the following: - A Kubernetes cluster, version 1.31 or later, with permission to create namespaces, deployments, and services. -- Cluster capacity for the Wave service's minimum compute requirements: - - Memory: 12 GB RAM (4 GB for each of two Wave pods, plus headroom for the cluster). - - CPU: 4 cores (1 core for each of two Wave pods, plus headroom for the cluster). - - Storage: 10 GB, plus disk space for your container images and temporary files. - - Network: Connectivity to your PostgreSQL and Redis instances. +- Cluster capacity for each Wave pod: the deployment below requests 2 GB RAM and 0.2 CPU per pod and limits it to 4 GB and 1 CPU. Scale that by your replica count. +- 10 GB storage, plus disk space for container images and temporary files. - PostgreSQL 16 or later, reachable from the cluster. - Redis 6.2 or later, reachable from the cluster. - A Seqera Platform deployment and its endpoint URL. - Access to the Wave container image from `cr.seqera.io`, using credentials provided by Seqera. ::: +:::tip +The manifests in this guide assemble into a single file. Save each YAML block into `wave.yaml` in the order shown, separated by `---`, then apply the file once at the end. +::: + :::tip[Install with the Helm chart] Seqera publishes an official [Wave Helm chart](https://artifacthub.io/packages/helm/seqera/wave) that deploys Wave Lite as an alternative to the raw manifests in this guide. Follow the chart's documentation to install it, and note the following: @@ -90,7 +89,7 @@ kubectl create secret docker-registry seqera-reg-creds \ ## Configure Wave -Create a ConfigMap with Wave's configuration. Update the database, Redis, and Platform values to match your environment. +Create a ConfigMap with Wave's configuration. Wave loads a single YAML document, so this is the whole of `config.yml` — add settings inside this block rather than appending a second `wave:` section. Update the database, Redis, Platform, and registry values to match your environment. :::warning This ConfigMap contains sensitive values. Use a Kubernetes Secret for credentials and reference it from the deployment rather than embedding secrets in the ConfigMap. See the [Kubernetes Secrets documentation](https://kubernetes.io/docs/concepts/configuration/secret/). @@ -122,43 +121,50 @@ data: uri: "jdbc:postgresql://postgres.example.com:5432/wave" user: "wave_user" password: "" + # One entry per private registry Wave pulls from. Public images need none. + registries: + docker.io: + username: "" + password: "" + quay.io: + username: "" + password: "" redis: # Use rediss:// for TLS (typical for managed Redis), or redis:// for a plain connection. uri: "rediss://redis.example.com:6379" tower: endpoint: url: "https://platform.example.com/api" + # Keep the JDBC and disk-space indicators out of /health. Micronaut enables them + # by default, and the liveness probe below would restart every pod on a brief + # database blip. + endpoints: + health: + enabled: true + disk-space: + enabled: false + jdbc: + enabled: false ``` :::warning Set `wave.server.url` to the address clients use to reach Wave. If you leave it unset, Wave issues container tokens pointing at `http://localhost:9090`, which clients cannot reach. ::: -This ConfigMap sets only what Wave Lite needs to start. The `lite` entry in `MICRONAUT_ENVIRONMENTS`, set in the deployment in a later step, already applies these same defaults. The ConfigMap restates them explicitly and gives you a place to add further configuration. To configure other options, such as rate limits, token cache duration, and metrics, see [Configure Wave](configure-wave.md). Before serving production traffic, complete the [production checklist](configure-wave.md#production-checklist). +The `lite` entry in `MICRONAUT_ENVIRONMENTS`, set in the deployment in a later step, already applies the four feature toggles. The ConfigMap restates them explicitly and gives you a place to add further configuration. For every available setting, see the [Configuration reference](reference.md). Before serving production traffic, complete the [production checklist](configure-wave.md#production-checklist). -## Authenticate to private registries +## Registry credentials -Wave Lite pulls images during augmentation. To augment images from a private registry, give Wave credentials for that registry. Wave uses one of two credential sources per request: +Wave Lite pulls images during augmentation, and uses one of two credential sources per request: - **Platform workspace credentials**: credentials a user adds to their Seqera Platform workspace. Wave uses these for requests that carry a Platform identity. -- **Server-side static credentials**: credentials the operator sets under `wave.registries.`. Wave uses these for anonymous requests and for registries the operator owns. +- **Server-side static credentials**: the `wave.registries.` entries in the ConfigMap above. Wave uses these for anonymous requests and for registries the operator owns. -Add an entry for each private registry under `wave.registries` in the `wave-cfg` config. For example, Docker Hub and a private Quay.io account: +For all registry options, see [Container registry](reference.md#container-registry). -```yaml -wave: - registries: - docker.io: - username: "" - password: "" - quay.io: - username: "" - password: "" -``` - -As with the database and Redis credentials, keep these out of the ConfigMap in production. Store them in a Kubernetes Secret and reference it from the deployment. - -Configure credentials for every private registry Wave pulls from. Public images need none. For all registry options, see [Container registry](reference.md#container-registry). +:::warning +Anonymous access is enabled by default, so any client that can reach Wave can use the operator credentials to pull through it. Disable it with `wave.capabilities.anonymous-access: false` before you expose the service — see [Require authentication](configure-wave.md#require-authentication). +::: ## Create the deployment @@ -253,7 +259,7 @@ spec: Wave must be reachable from Seqera Platform and from your Nextflow compute environments. Front the service with an ingress and terminate TLS at the ingress or load balancer. Wave does not terminate TLS itself. -This example uses an AWS ALB. For NGINX or GCE ingress, see [Adapt this guide](#adapt-this-guide). +This example uses the AWS Load Balancer Controller. `target-type: ip` is what lets it route to the `ClusterIP` service defined above — with the default `instance` target type, change that service to `NodePort`. Replace the certificate ARN with your own: ```yaml apiVersion: networking.k8s.io/v1 @@ -261,7 +267,14 @@ kind: Ingress metadata: name: wave-ingress namespace: wave + annotations: + alb.ingress.kubernetes.io/scheme: internet-facing + alb.ingress.kubernetes.io/target-type: ip + alb.ingress.kubernetes.io/listen-ports: '[{"HTTPS":443}]' + alb.ingress.kubernetes.io/certificate-arn: "arn:aws:acm:::certificate/" + alb.ingress.kubernetes.io/healthcheck-path: /health spec: + ingressClassName: alb rules: - host: wave.example.com http: @@ -275,25 +288,21 @@ spec: number: 9090 ``` -:::note -This minimal Ingress omits controller-specific configuration. For the AWS Load Balancer Controller, add `ingressClassName: alb` and the `alb.ingress.kubernetes.io/*` annotations (scheme, target type, and ACM certificate ARN) your setup requires. With `alb.ingress.kubernetes.io/target-type: ip`, the `ClusterIP` service defined earlier works as-is. With the default `instance` target type, change the service to `NodePort`. -::: +For the certificate and the DNS record that points `wave.example.com` at the load balancer, see [Terminate TLS](configure-wave.md#terminate-tls). For NGINX, GCE, or Traefik, swap `ingressClassName` and the annotations for that controller's equivalents. -After the ingress provisions, configure your Seqera Platform deployment to use the Wave endpoint by setting the Wave server URL in `tower.yml` ([Platform Wave configuration](https://docs.seqera.io/platform-enterprise/latest/enterprise/configuration/wave)). +## Apply the manifests -:::note -For production reliability, add Pod Disruption Budgets, a Horizontal Pod Autoscaler, multiple replicas with anti-affinity, and resource quotas for the `wave` namespace. See the [production checklist](configure-wave.md#production-checklist). -::: +Apply the assembled file and wait for the rollout: -## Verify your installation - -Confirm the service is live and functional. See [Verify your installation](post-install.md) for the `/service-info` check and the Wave CLI functional checks. +```bash +kubectl apply -f wave.yaml +kubectl rollout status deployment/wave -n wave +``` -When Wave is running and verified, continue to the [production checklist](configure-wave.md#production-checklist) to prepare the deployment for production. +Then configure your Seqera Platform deployment to use the Wave endpoint by setting the Wave server URL in `tower.yml` ([Platform Wave configuration](https://docs.seqera.io/platform-enterprise/latest/enterprise/configuration/wave)). -## Adapt this guide +## Verify your installation -The supported procedure uses managed PostgreSQL and Redis and an AWS ALB ingress. The following options are described but not part of the procedure. Adapt them at your own risk. +Confirm the service is live and functional. See [Verify your installation](post-install.md) for the `/service-info` check and the Wave CLI functional checks. -- **Other ingress controllers**: NGINX, GCE, or Traefik work, but add the provider-specific annotations they require and verify TLS termination. -- **Other distributions**: Wave Lite has no AWS dependency and runs on any conformant Kubernetes distribution, but other distributions are not validated. +When Wave is running and verified, continue to the [production checklist](configure-wave.md#production-checklist) to prepare the deployment for production. Wave Lite has no AWS dependency and runs on any conformant Kubernetes distribution, though only EKS is validated — and only EKS can be extended to the full Wave configuration. diff --git a/docs/install/post-install.md b/docs/install/post-install.md index 3354c9cea..bd5c1e7b4 100644 --- a/docs/install/post-install.md +++ b/docs/install/post-install.md @@ -43,7 +43,7 @@ export WAVE_ENDPOINT=https://wave.example.com ``` :::note -If you disabled anonymous access in [Configure Wave](configure-wave.md#require-authentication), the CLI checks need a Seqera Platform access token. Pass it with the `--tower-token` flag or the `TOWER_ACCESS_TOKEN` environment variable. +If you disabled anonymous access in [Configure Wave](configure-wave.md#require-authentication), every CLI check needs a Seqera Platform access token. Pass it with the `--tower-token` flag or the `TOWER_ACCESS_TOKEN` environment variable. ::: ## Functional checks @@ -77,6 +77,20 @@ wave --containerfile Dockerfile --freeze --build-repo --tower-token A successful build returns a reference in your configured build repository. +If you enabled mirroring, copy an image into your build repository. `--mirror` requires a build repository and cannot be combined with `--containerfile`, `--conda-package`, or `--freeze`: + +```bash +wave -i ubuntu:22.04 --mirror --build-repo +``` + +If you enabled scanning, request a build that must pass a scan before it is returned: + +```bash +wave --conda-package bcftools --scan-mode required +``` + +A scan failure returns the vulnerabilities found rather than an image reference. Use `--scan-level` to set which severities are tolerated. + :::note Freeze builds push to the repository you name. They always need a Platform access token, and the associated Platform workspace must hold registry credentials with push access to ``. See [Freeze and user-supplied build repositories](aws-build.md#freeze-and-user-supplied-build-repositories). ::: diff --git a/docs/install/reference.md b/docs/install/reference.md index 1b4f45eb3..471ce5485 100644 --- a/docs/install/reference.md +++ b/docs/install/reference.md @@ -27,11 +27,7 @@ Configure general Wave application settings. : URL of the Seqera Platform API service (default: [`https://api.cloud.seqera.io`](https://api.cloud.seqera.io)). Can be set using the `${TOWER_ENDPOINT_URL}` environment variable. -`wave.allowAnonymous` *(required)* -: When `true`, anonymous users can access the Wave server (default: `true`). - Set to `false` in production so that every request must carry a Platform-issued token. - -`wave.denyHosts` *(optional)* +`wave.deny-hosts` *(optional)* : Hostname patterns to deny. Requests targeting these hosts are rejected. Example patterns: `ngrok.app`, `ngrok-free.app`, `//localhost`. @@ -66,13 +62,30 @@ Configure general Wave application settings. `wave.tokens.watcher.count` *(optional)* : Maximum number of container requests processed in a single watcher cycle (default: `250`). +## Feature toggles + +Each Wave feature is an independent toggle. Wave Lite runs with all four off, which is what the `lite` Micronaut environment applies. + +`wave.build.enabled` *(optional)* +: When `true`, Wave provisions containers with on-demand builds (default: `true`). Freeze and scanning both depend on the build pipeline and are unavailable when this is `false`. + +`wave.mirror.enabled` *(optional)* +: When `true`, Wave can mirror images into a target repository (default: `true`). + +`wave.scan.enabled` *(optional)* +: When `true`, activates vulnerability scanning (default: `false`). Requires `wave.build.enabled` and `wave.scan.reports.path`. + +`wave.blob-cache.enabled` *(optional)* +: When `true`, Wave caches container layers in object storage (default: `false`). Requires the blob cache storage settings. + ## Capabilities The `wave.capabilities.*` flags are installation-level toggles for optional Wave capabilities. All default to `true` (permissive). Set a flag to `false` to lock down that capability, or enable the `strict` environment (`MICRONAUT_ENVIRONMENTS=strict`) to disable all of them at once — intended for regulated deployments where Wave must not serve images directly, broker credentials, expose its HTML pages, or accept anonymous requests. `wave.capabilities.anonymous-access` *(optional)* : When `true`, anonymous (unauthenticated) users can access the Wave server (default: `true`). - Set to `false` to require authenticated access. Modify this option based on your security requirements. + Set to `false` to require authenticated access, so that every request must carry a Platform-issued token. + Legacy alias: `wave.allowAnonymous`. Prefer the canonical key; the alias is still honored. `wave.capabilities.ephemeral-token` *(optional)* : When `true`, Wave can provision a container by pulling an existing image directly, applying any container configuration on the fly (the augmentation path) (default: `true`). @@ -93,6 +106,8 @@ You must specify all repositories used in your Wave installation. The following examples show standard formats for known registries, but you can customize the registry name (for example, change `azurecr.io` to `seqeralabs.azurecr.io`). +The environment variables noted below are not read directly by Wave: they take effect only where your `config.yml` interpolates them, as in `username: "${DOCKER_USER:}"`. Setting `DOCKER_USER` without such an entry has no effect. + Configure container registry authentication with the following options. `wave.registries..password` *(optional)* @@ -124,6 +139,20 @@ Configure container registry authentication with the following options. : Quay.io username for authentication. Can be set using the `${QUAY_USER}` environment variable. +## Registry pre-creation + +Wave pushes with BuildKit for builds and Skopeo for mirrors, so whether a target repository must exist beforehand is the registry's rule, not Wave's. If the registry requires pre-creation and the path is missing, the push fails partway through the layer upload. See [Registry push and authentication failures](../troubleshoot.md#registry-push-and-authentication-failures) to diagnose one. + +| Registry | Pre-creation | Notes | +| --- | --- | --- | +| Amazon ECR | Required | Every repository must exist before push. Registry-level auto-create policies exist but are off by default. | +| Docker Hub | Not required | Repositories auto-create in your user or organization namespace. Repository-count and pull rate limits apply. | +| GitHub Container Registry | Not required | Auto-creates under the user or organization namespace; visibility inherits from the organization's package settings. | +| Google Artifact Registry | Partial | Create the repository with `gcloud artifacts repositories create`; image paths inside it auto-create. | +| Google Container Registry | Not required | Auto-creates on push. Being phased out — target Artifact Registry for new deployments. | +| Azure Container Registry | Partial | The ACR instance must exist; image paths inside it auto-create. Wave needs the `AcrPush` role. | +| Harbor | Partial | Create the project through the UI or API; images inside it auto-create if project policy permits. | + ## AWS cross-account role chaining To access ECR registries in customer AWS accounts using IAM role credentials from the Seqera Platform, configure an intermediate "jump role" for cross-account access. Wave first assumes the jump role using its own credentials, then uses the jump role's temporary credentials to assume the target role from the Seqera Platform. @@ -208,7 +237,7 @@ Configure the HTTP client with the following options. Configure how Wave builds container images and manages build logs. -`wave.build.buildkit-image` *(required)* +`wave.build.buildkit-image` *(optional)* : [Buildkit](https://github.com/moby/buildkit) container image used in the Wave build process (default: `public.cr.seqera.io/wave/buildkit:v0.25.2-rootless`). `wave.build.cache` *(optional)* @@ -273,7 +302,7 @@ Configure how Wave builds container images and manages build logs. For example, `/efs/wave/build`. :::tip -For S3 cache authentication setup (IAM roles, service accounts, and deployment examples), see [S3 cache authentication](configure-wave.md#s3-cache-authentication). +For S3 cache authentication setup, see [Build layer cache](configure-wave.md#build-layer-cache). ::: ### Cleanup @@ -369,9 +398,6 @@ Configure Kubernetes-specific settings for Wave. Build and scan processes share Configure Wave's vulnerability scanning process, which uses a [Trivy Docker image](https://hub.docker.com/r/aquasec/trivy) with customizable tags and severity levels. -`wave.scan.enabled` *(optional)* -: When `true`, activates vulnerability scanning (default: `false`). - `wave.scan.image.name` *(optional)* : Container image used for security scanning (default: `public.cr.seqera.io/wave/scanner:v1-0.65.0-oras-1.3.0`). @@ -435,16 +461,16 @@ Configure rate limits for anonymous and authenticated user access. These options take effect only when the `rate-limit` entry is included in the `MICRONAUT_ENVIRONMENTS` variable of your deployment. ::: -`rate-limit.build.anonymous` *(required)* +`rate-limit.build.anonymous` *(optional)* : Rate limit for build requests from anonymous users (default: `10/1h`). -`rate-limit.build.authenticated` *(required)* +`rate-limit.build.authenticated` *(optional)* : Rate limit for build requests from authenticated users (default: `10/1m`). -`rate-limit.pull.anonymous` *(required)* +`rate-limit.pull.anonymous` *(optional)* : Rate limit for pull requests from anonymous users (default: `100/1h`). -`rate-limit.pull.authenticated` *(required)* +`rate-limit.pull.authenticated` *(optional)* : Rate limit for pull requests from authenticated users (default: `100/1m`). `rate-limit.timeout-errors.max-rate` *(optional)* @@ -557,7 +583,7 @@ Configure how Wave caches container blobs to improve client performance. Wave ca `wave.blob-cache.k8s.resources.limits.memory` *(optional)* : Memory resource [limit](https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/#resource-units-in-kubernetes) for the Kubernetes pod used for blob binary transfers. -`wave.blobCache.s5cmdImage` *(optional)* +`wave.blob-cache.s5cmd-image` *(optional)* : Container image that supplies the [s5cmd tool](https://github.com/peak/s5cmd) for uploading blob binaries to the S3 bucket (default: `public.cr.seqera.io/wave/s5cmd:v2.3.0`). `wave.blob-cache.signing-strategy` *(optional)* @@ -571,20 +597,20 @@ Configure how Wave caches container blobs to improve client performance. Wave ca `wave.blob-cache.status.duration` *(optional)* : Duration for which blob transfer status records are retained in cache (default: `1h`). -`wave.blobCache.storage.accessKey` *(required when blob cache is enabled)* +`wave.blob-cache.storage.access-key` *(required when blob cache is enabled)* : Access key credential for the caching service. -`wave.blobCache.storage.bucket` *(required when blob cache is enabled)* +`wave.blob-cache.storage.bucket` *(required when blob cache is enabled)* : Name of the Cloudflare or S3 bucket. For example, `s3://wave-blob-cache`. `wave.blob-cache.storage.endpoint` *(optional)* : Storage endpoint URL for blob binary downloads and uploads. -`wave.blobCache.storage.region` *(required when blob cache is enabled)* +`wave.blob-cache.storage.region` *(required when blob cache is enabled)* : AWS region of the bucket. -`wave.blobCache.storage.secretKey` *(required when blob cache is enabled)* +`wave.blob-cache.storage.secret-key` *(required when blob cache is enabled)* : Secret key credential for the caching service.
@@ -602,10 +628,36 @@ Static credentials (`access-key` and `secret-key`) are currently required for bl Configure how Wave sends email notifications. +Email delivery requires `mail` in `MICRONAUT_ENVIRONMENTS`. Add `aws-ses` as well to send through Amazon SES with IAM authentication, in which case only `mail.from` applies and the `mail.smtp.*` settings are ignored. + `mail.from` *(required when mail is enabled)* : Sender email address for Wave notifications. Can be set using the `${MAIL_FROM}` environment variable. +`mail.smtp.host` *(required when SMTP is used)* +: SMTP server hostname. + +`mail.smtp.port` *(required when SMTP is used)* +: SMTP server port, typically `587` for STARTTLS or `465` for implicit TLS. + +`mail.smtp.user` *(optional)* +: Username for SMTP authentication. + +`mail.smtp.password` *(optional)* +: Password for SMTP authentication. + +`mail.smtp.auth` *(optional)* +: When `true`, authenticate to the SMTP server. + +`mail.smtp.starttls.enable` *(optional)* +: When `true`, upgrade the connection with STARTTLS. + +`mail.smtp.starttls.required` *(optional)* +: When `true`, fail rather than fall back to an unencrypted connection. + +`mail.smtp.ssl.protocols` *(optional)* +: Space-separated list of permitted TLS protocols, for example `TLSv1.2 TLSv1.3`. + ## Metrics Configure the Wave Metrics service, which provides data about container builds and pulls per organization and date. @@ -617,10 +669,21 @@ Configure the Wave Metrics service, which provides data about container builds a Configure user credentials for accessing authenticated Wave APIs and services. -`wave.accounts` *(required)* +`wave.accounts` *(optional)* : Credentials for accessing authenticated Wave APIs such as the metrics API. A map of usernames to SHA-256 hex checksums of the corresponding passwords: +
+ + ```yaml + wave: + accounts: + # SHA-256 checksum of the password 'bar' + foo: "fcde2b2edba56bf408601fb721fe9b5c338d10ee429ea04fae5511b68fbf8fb9" + ``` + +
+ ## License server Configure the connection to the Seqera license management server: @@ -634,6 +697,10 @@ Configure the connection to the Seqera license management server: Configure polling and scheduling intervals for Wave's asynchronous job processing. +`wave.job-manager.max-running-jobs` *(optional)* +: Maximum number of build, scan, and mirror jobs Wave runs concurrently (default: `20`). + Pair this with a `ResourceQuota` on the build namespace to bound build resource usage. + `wave.job-manager.poll-interval` *(optional)* : Polling interval for checking job status (default: `1s`). diff --git a/docs/sidebar.json b/docs/sidebar.json index eb09c2ef0..f521d3daf 100644 --- a/docs/sidebar.json +++ b/docs/sidebar.json @@ -49,8 +49,8 @@ "install/kubernetes-lite" ] }, - "install/aws-build", "install/post-install", + "install/aws-build", "install/configure-wave", "install/reference" ] @@ -68,6 +68,7 @@ ] }, "faq", + "troubleshoot", { "type": "link", "label": "Changelog", diff --git a/docs/troubleshoot.md b/docs/troubleshoot.md index 9e445f2ea..f835b4d38 100644 --- a/docs/troubleshoot.md +++ b/docs/troubleshoot.md @@ -1,3 +1,8 @@ +--- +title: Troubleshoot +description: Diagnose common Wave build, registry, and node configuration failures. +--- + ## Troubleshoot guide 1. How to troubleshoot container build failure? @@ -29,8 +34,48 @@ sh: can't kill pid 14: No such process When Wave cannot push a built or mirrored image, or cannot authenticate to a registry, the failure usually matches one of these symptoms: - **No credentials match the target host.** Wave returns an authentication error at token-request time, before BuildKit or Skopeo launches. This is the fastest failure to diagnose. Confirm you configured credentials for the target registry. -- **The repository does not exist and the registry requires pre-creation.** The push fails with `403 Forbidden` or `404 Not Found` partway through the layer upload, often after an initial `HEAD` succeeds but the final manifest `PUT` fails. Pre-create the repository. For per-registry rules, see [per-registry pre-creation rules](install/aws-build.md#create-the-ecr-repositories). +- **The repository does not exist and the registry requires pre-creation.** The push fails with `403 Forbidden` or `404 Not Found` partway through the layer upload, often after an initial `HEAD` succeeds but the final manifest `PUT` fails. Pre-create the repository. For per-registry rules, see [Registry pre-creation](install/reference.md#registry-pre-creation). - **Credentials exist but lack push scope.** The push typically returns a `403` on the final manifest `PUT` even though layer uploads appear to work. Check the credential's scope for `push`, `write`, or `deploy` permission. - **The repository key is missing from the path (JFrog Artifactory).** The push fails with `404`. Confirm the repository key is the first path segment after the host, for example `artifactory.example.com/docker-local/...`. - **The registry exists but the configured AWS credentials or role target the wrong region or account.** For ECR, a misaligned `aws.region` or jump-role configuration produces STS `AccessDenied` errors in the Wave service logs. These errors appear when Wave exchanges the configured credentials for an ECR auth token, before the build pod launches. +## Builds fail on Bottlerocket nodes + +BuildKit requires user namespaces, and Bottlerocket sets `user.max_user_namespaces=0` by default, so every build fails on an otherwise correct [Wave build deployment](install/aws-build.md). Raise the limit on your build nodes — `63359` is a reasonable value. Values that are too low cap concurrent build capacity and cause intermittent failures. + +Set it at boot through the node group's startup script or user data. That is the preferred route: it needs no privileged containers. If you cannot change node configuration directly, apply it with a DaemonSet scoped to the build nodes: + +```yaml +apiVersion: apps/v1 +kind: DaemonSet +metadata: + labels: + app: sysctl-userns + name: sysctl-userns +spec: + selector: + matchLabels: + app: sysctl-userns + template: + metadata: + labels: + app: sysctl-userns + spec: + containers: + - name: sysctl-userns + image: busybox + command: ["sh", "-euxc", "sysctl -w user.max_user_namespaces=63359 && sleep infinity"] + securityContext: + privileged: true + affinity: + nodeAffinity: + requiredDuringSchedulingIgnoredDuringExecution: + nodeSelectorTerms: + - matchExpressions: + - key: service + operator: In + values: ["wave-build", "wave-build-arm64"] +``` + +For more on Bottlerocket settings, see the [Bottlerocket FAQs](https://bottlerocket.dev/en/faq/). +