Skip to content

perf(ci): parallelize e2e operator build prerequisites #993

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
bennyz merged 3 commits into from
Aug 12, 2026
+74 −4
Merged

perf(ci): parallelize e2e operator build prerequisites #993

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
39f498a
perf(ci): parallelize e2e operator build prerequisites
bennyz Aug 10, 2026
5d60cea
perf(ci): compile Go on host with cached modules instead of inside co…
bennyz Aug 10, 2026
64cd0e7
perf(ci): apply host-compiled Go to deploy-kind job
bennyz Aug 10, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 21 additions & 2 deletions .github/workflows/controller-kind.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,9 @@ on:
paths:
- 'controller/**'

permissions:
contents: read

jobs:
deploy-kind:
runs-on: ubuntu-latest
Expand All @@ -15,9 +18,17 @@ jobs:
with:
fetch-depth: 0

- name: Set up Go
uses: actions/setup-go@d35c59abb061a4a6fb18e82ac0862c26744d6ab5 # v5
with:
go-version-file: controller/go.mod
cache-dependency-path: |
controller/go.sum
controller/deploy/operator/go.sum

- name: Run make deploy
working-directory: controller
run: make deploy
run: make deploy-ci -j4 --output-sync=target

e2e-test-operator:
runs-on: ubuntu-latest
Expand All @@ -27,6 +38,14 @@ jobs:
with:
fetch-depth: 0

- name: Set up Go
uses: actions/setup-go@d35c59abb061a4a6fb18e82ac0862c26744d6ab5 # v5
with:
go-version-file: controller/go.mod
cache-dependency-path: |
controller/go.sum
controller/deploy/operator/go.sum

- name: Run operator e2e test
working-directory: controller
run: make test-operator-e2e
run: make test-operator-e2e-ci -j4 --output-sync=target
13 changes: 13 additions & 0 deletions controller/Containerfile.prebuilt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
# CI-only runtime image for host-compiled Go binaries.
#
# BIN selects which binary becomes the default entrypoint.
# COPY ${BIN} /entrypoint works because COPY supports ARG expansion,
# while ENTRYPOINT does not (and ubi-micro has no shell for shell-form).
FROM registry.access.redhat.com/ubi9/ubi-micro:9.8-1784702951@sha256:b1e86b97028b8fcfb6d85f997c39e6b6b67496163ef8d80d243220a4918e8bef
ARG BIN
WORKDIR /
COPY . .
# ENTRYPOINT can't expand ARG; fixed path lets one file serve all three images
COPY ${BIN} /entrypoint
Comment on lines +9 to +11

@bkhizgiy bkhizgiy Aug 12, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

COPY . . copies everything from the build context, including the binaries, since ${BIN} is copied again as /entrypoint, we end up shipping duplicate manager/router binaries that aren't used.
So it probably can be simplified to just COPY ${BIN} /entrypoint

@bennyz bennyz Aug 12, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

coderabbit commented on this too,
the controller image has manager and router, and we have a third exporter-set-controller

so in order to get router too we copy it in COPY . .
at the expense of an extra 40-50 throwaway MBs

I'll add a comment to clarify

USER 65532:65532
ENTRYPOINT ["/entrypoint"]
Comment on lines +11 to +13

@coderabbitai coderabbitai Bot Aug 12, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟡 Minor | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

rg -n -C 3 '(/manager|/entrypoint)' controller

Repository: jumpstarter-dev/jumpstarter

Length of output: 7688

🏁 Script executed:

#!/bin/bash
set -euo pipefail

printf '%s\n' '--- Containerfiles ---'
cat -n controller/Containerfile.prebuilt
cat -n controller/Containerfile.operator

printf '%s\n' '--- Operator manager manifest ---'
cat -n controller/deploy/operator/config/manager/manager.yaml

printf '%s\n' '--- Relevant build targets ---'
sed -n '135,152p' controller/Makefile
sed -n '176,187p' controller/deploy/operator/Makefile

printf '%s\n' '--- All operator command/entrypoint/probe references ---'
rg -n -C 4 'command:|/manager|/entrypoint|livenessProbe|readinessProbe|startupProbe' controller/deploy/operator controller/Containerfile.prebuilt controller/Containerfile.operator

Repository: jumpstarter-dev/jumpstarter

Length of output: 18058

🏁 Script executed:

#!/bin/bash
set -euo pipefail

python3 - <<'PY'
from pathlib import Path
import re

prebuilt = Path("controller/Containerfile.prebuilt").read_text()
operator = Path("controller/Containerfile.operator").read_text()
manager = Path("controller/deploy/operator/config/manager/manager.yaml").read_text()

copy_all = bool(re.search(r"^COPY\s+\.\s+\.$", prebuilt, re.M))
copy_bin = re.search(r"^COPY\s+\$\{BIN\}\s+(\S+)$", prebuilt, re.M)
prebuilt_entrypoint = re.search(r"^ENTRYPOINT\s+(.+)$", prebuilt, re.M)
operator_entrypoint = re.search(r"^ENTRYPOINT\s+(.+)$", operator, re.M)
manager_command = re.search(r"^\s*-\s+(/(?:manager|entrypoint))\s*$", manager, re.M)

print({
    "prebuilt_COPY_dot_dot": copy_all,
    "prebuilt_binary_destination": copy_bin.group(1) if copy_bin else None,
    "prebuilt_entrypoint": prebuilt_entrypoint.group(1) if prebuilt_entrypoint else None,
    "operator_entrypoint": operator_entrypoint.group(1) if operator_entrypoint else None,
    "generated_manager_command": manager_command.group(1) if manager_command else None,
    "prebuilt_contains_manager_via_context_copy": copy_all,
})
PY

Repository: jumpstarter-dev/jumpstarter

Length of output: 417

Remove the duplicate binary copy.

COPY . . already copies ${BIN} to /${BIN}, while the next instruction copies it again to /entrypoint. The deployment explicitly runs /manager, so keep one runtime path and update all consumers consistently.

🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@controller/Containerfile.prebuilt` around lines 5 - 7, Remove the redundant
binary copy to /entrypoint in the Containerfile, retain the existing ${BIN} path
copied by COPY . ., and update ENTRYPOINT and any related runtime references to
consistently execute /${BIN} (the deployed /manager path).

34 changes: 32 additions & 2 deletions controller/Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,8 @@ else
GOBIN=$(shell go env GOBIN)
endif

GOARCH ?= $(shell go env GOARCH)

# CONTAINER_TOOL defines the container tool to be used for building images.
# Be aware that the target commands are only tested with Docker which is
# scaffolded by default. However, you might want to replace it to use other
Expand Down Expand Up @@ -108,7 +110,13 @@ lint-fix: golangci-lint ## Run golangci-lint linter and perform fixes
##@ Build
.PHONY: build-operator
build-operator:
make -C deploy/operator build-installer docker-build
$(MAKE) -C deploy/operator build-installer
$(MAKE) -C deploy/operator docker-build

.PHONY: build-operator-ci
build-operator-ci:
$(MAKE) -C deploy/operator build-installer
$(MAKE) -C deploy/operator docker-build-ci

.PHONY: build
build: manifests generate fmt vet ## Build manager binary.
Expand All @@ -135,6 +143,15 @@ docker-build: ## Build docker image with the manager.
--build-arg BUILD_DATE=$(BUILD_DATE) \
-t ${IMG} -f Containerfile .

.PHONY: docker-build-ci
docker-build-ci: ## Build docker images from pre-compiled host binaries (fast CI path).
rm -rf bin/ci-stage && mkdir -p bin/ci-stage/controller bin/ci-stage/esc
CGO_ENABLED=0 GOOS=linux GOARCH=$(GOARCH) go build -ldflags "$(LDFLAGS)" -o bin/ci-stage/controller/manager cmd/main.go
CGO_ENABLED=0 GOOS=linux GOARCH=$(GOARCH) go build -ldflags "$(LDFLAGS)" -o bin/ci-stage/controller/router cmd/router/main.go
CGO_ENABLED=0 GOOS=linux GOARCH=$(GOARCH) go build -ldflags "$(LDFLAGS)" -o bin/ci-stage/esc/exporter-set-controller cmd/exporter-set-controller/main.go
$(CONTAINER_TOOL) build --build-arg BIN=manager -t $(IMG) -f Containerfile.prebuilt bin/ci-stage/controller
$(CONTAINER_TOOL) build --build-arg BIN=exporter-set-controller -t $(EXPORTER_SET_CONTROLLER_IMG) -f Containerfile.prebuilt bin/ci-stage/esc

.PHONY: docker-build-exporter-set-controller
docker-build-exporter-set-controller: ## Build docker image for the exporter-set-controller.
$(CONTAINER_TOOL) build \
Expand Down Expand Up @@ -206,14 +223,27 @@ ifeq ($(SKIP_BUILD),)
endif
./hack/deploy_with_operator.sh

.PHONY: deploy-ci
deploy-ci: cluster grpcurl $(if $(SKIP_BUILD),,docker-build-ci build-operator-ci) ## CI-optimized deploy: host-compiled binaries, no multi-stage container builds.
./hack/deploy_with_operator.sh
Comment thread
coderabbitai[bot] marked this conversation as resolved.


.PHONY: deploy-operator
deploy-operator: docker-build docker-build-exporter-set-controller build-operator cluster grpcurl ## Deploy only the operator (without Jumpstarter CR)
NETWORKING_MODE=ingress DEPLOY_JUMPSTARTER=false ./hack/deploy_with_operator.sh

.PHONY: deploy-operator-ci
deploy-operator-ci: docker-build-ci build-operator-ci cluster grpcurl ## CI-optimized: host-compiled binaries, no multi-stage container builds.
NETWORKING_MODE=ingress DEPLOY_JUMPSTARTER=false ./hack/deploy_with_operator.sh

.PHONY: test-operator-e2e
test-operator-e2e: grpcurl deploy-operator
make -C deploy/operator test-e2e
$(MAKE) -C deploy/operator test-e2e

.PHONY: test-operator-e2e-ci
test-operator-e2e-ci: grpcurl deploy-operator-ci ## CI-optimized e2e test (host-compiled Go, cached modules).
$(MAKE) -C deploy/operator test-e2e

.PHONY: operator-logs
operator-logs:
kubectl logs -n jumpstarter-operator-system -l app.kubernetes.io/name=jumpstarter-operator -f
Expand Down
8 changes: 8 additions & 0 deletions controller/deploy/operator/Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,8 @@ else
GOBIN=$(shell go env GOBIN)
endif

GOARCH ?= $(shell go env GOARCH)

Comment on lines +74 to +75

@coderabbitai coderabbitai Bot Aug 12, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -eu

printf '%s\n' '--- Makefile context ---'
sed -n '1,230p' controller/deploy/operator/Makefile

printf '%s\n' '--- Containerfile.prebuilt locations and contents ---'
fd -i 'Containerfile.prebuilt' .
for f in $(fd -i 'Containerfile.prebuilt' .); do
  echo "### $f"
  cat -n "$f"
done

printf '%s\n' '--- architecture-related references ---'
rg -n --glob 'Makefile' --glob 'Containerfile*' --glob '*.yaml' --glob '*.yml' \
  'GOARCH|GOARM|TARGET_PLATFORM|platform=|Containerfile\.prebuilt|bin/ci-stage|entrypoint' .

Repository: jumpstarter-dev/jumpstarter

Length of output: 15697

🏁 Script executed:

#!/bin/bash
set -eu

python3 - <<'PY'
from pathlib import Path
import re

makefile = Path("controller/deploy/operator/Makefile").read_text()
containerfile = Path("controller/Containerfile.prebuilt").read_text()

target = re.search(
    r"docker-build-ci:.*?\n(?P<body>(?:\t.*\n)+)",
    makefile,
)
assert target, "docker-build-ci target not found"
body = target.group("body")

assert "GOARCH=$(GOARCH)" in body, "docker-build-ci does not compile with GOARCH"
build = next(
    line for line in body.splitlines()
    if "$(CONTAINER_TOOL) build" in line
)
assert "--platform" not in build, "build command already specifies --platform"

assert re.search(r"^FROM\s+(?!--platform=)", containerfile, re.M), \
    "Containerfile.prebuilt unexpectedly sets a FROM platform"
assert "COPY ${BIN} /entrypoint" in containerfile, \
    "Containerfile.prebuilt does not copy the prebuilt binary"
assert "ENTRYPOINT [\"/entrypoint\"]" in containerfile, \
    "Containerfile.prebuilt does not execute the copied binary"

print("docker-build-ci compiles with configurable GOARCH")
print("its container build has no --platform option")
print("Containerfile.prebuilt has no FROM --platform override")
print("the copied binary is the image entrypoint")
PY

printf '%s\n' '--- docker-build-ci invocations and GOARCH overrides ---'
rg -n -C 3 'docker-build-ci|GOARCH=|make .*docker-build-ci' \
  .github controller --glob '*.yaml' --glob '*.yml' --glob 'Makefile' \
  --glob '*.mk' || true

Repository: jumpstarter-dev/jumpstarter

Length of output: 3823

Set the image platform from GOARCH.

When GOARCH differs from the host, docker-build-ci creates a target-architecture /entrypoint, but the container build defaults to the host platform. The image can advertise the host architecture and fail with exec format error.

Proposed fix 
 GOARCH ?= $(shell go env GOARCH)
+TARGET_PLATFORM ?= linux/$(GOARCH)

 ...

-	$(CONTAINER_TOOL) build --build-arg BIN=manager -t $(IMG) -f ../../Containerfile.prebuilt bin/ci-stage
+	$(CONTAINER_TOOL) build --platform=$(TARGET_PLATFORM) --build-arg BIN=manager -t $(IMG) -f ../../Containerfile.prebuilt bin/ci-stage

Map variants such as GOARM if the repository supports them.

🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@controller/deploy/operator/Makefile` around lines 74 - 75, Update the
docker-build-ci image build flow to pass the target platform derived from GOARCH
to the container build, rather than allowing the host platform default; include
GOARM or other supported architecture variants in the mapping so the image
platform matches the generated /entrypoint architecture.

# CONTAINER_TOOL defines the container tool to be used for building images.
# Be aware that the target commands are only tested with Docker which is
# scaffolded by default. However, you might want to replace it to use other
Expand Down Expand Up @@ -177,6 +179,12 @@ docker-build: ## Build docker image with the manager.
--build-arg BUILD_DATE=$(BUILD_DATE) \
-t ${IMG} ../../ -f ../../Containerfile.operator

.PHONY: docker-build-ci
docker-build-ci: ## CI-optimized: host-compiled binary, no multi-stage build.
rm -rf bin/ci-stage && mkdir -p bin/ci-stage
CGO_ENABLED=0 GOOS=linux GOARCH=$(GOARCH) go build -ldflags "$(LDFLAGS)" -o bin/ci-stage/manager cmd/main.go
$(CONTAINER_TOOL) build --build-arg BIN=manager -t $(IMG) -f ../../Containerfile.prebuilt bin/ci-stage

.PHONY: docker-push
docker-push: ## Push docker image with the manager.
$(CONTAINER_TOOL) push ${IMG}
Expand Down
Loading