Repository navigation
Image publishes have failed since 2026-09-08: Cargo.lock on main is unusable with --locked #155
Description
Every GHCR publish of doublezero-edge-connect has failed since 2026-09-08. As a result ghcr.io/malbeclabs/doublezero-edge-connect:mainnet-beta is still the 2026-09-07 build, pinned to the 0.38.0 client base — it missed the doublezero v0.39.0 release and the doublezero:mainnet-beta-0.39.0-1 base image published on 2026-09-10.
The fix is already open and green: #151. It has been waiting on review since 2026-09-07.
Symptom
Both the event-driven rebuild and the daily poll backstop fail in the Docker build step:
#20 0.570 error: cannot update the lock file /src/Cargo.lock because --locked was passed to prevent this
ERROR: failed to build: failed to solve: process "/bin/sh -c cargo build --release --locked -p doublezero-edge-connect ..." did not complete successfully: exit code: 101
Failing runs (all the same error):
| When | Workflow | Run |
|---|---|---|
| 2026-09-08 14:16 | dispatch | 34237239761 |
| 2026-09-09 05:30 | poll | 34315124137 |
| 2026-09-10 05:30 | poll | 34441265726 |
| 2026-09-10 05:43 | dispatch (mainnet-beta 0.39.0) | 34442208828 |
| 2026-09-10 15:10 | dispatch | 34493872900 |
The upstream side is healthy — releaser.client for client/v0.39.0, the doublezero base image publishes and the cross-repo repository_dispatch all succeeded. The failure is entirely here.
Root cause
Six Dependabot PRs merged into main within about 30 seconds on 2026-09-07 without being rebased onto one another:
- chore(deps): bump ed25519-dalek from 2.2.0 to 3.0.0 #85 (ed25519-dalek 2.2.0 → 3.0.0) resolved a lock with a single
rand_coreand dropped the olddigest 0.10chain. - chore(deps): bump the rust-minor-patch group across 1 directory with 7 updates #81 (rust-minor-patch group, lock-only) was resolved against the pre-upgrade tree.
Each lock was correct against the base it was resolved against, and git merged them line by line with no conflict. The result on main (e258968) is a lock Cargo cannot use:
- 2 ambiguous dependency refs —
rand 0.9.4andrand_chacha 0.9.0name a bare"rand_core"while bothrand_core 0.9.5andrand_core 0.10.1are locked. Cargo cannot resolve a bare name against two versions, so it must rewrite the lock, which--lockedforbids. - 6 orphan entries reachable from nothing, resurrected by the merge:
block-buffer 0.10.4,cpufeatures 0.2.17,crypto-common 0.1.7,digest 0.10.7,generic-array 0.14.7,version_check 0.9.5.
Reproduces at e258968:
$ cargo metadata --locked --format-version 1
error: cannot update the lock file ... because --locked was passed to prevent this
Why CI didn't catch it
.github/workflows/rust.yml runs cargo build/clippy/test --workspace without --locked, so it silently re-resolves and stays green. The Dockerfile's cargo build --release --locked (Dockerfile:50) is the only enforcement — so the first symptom is a failed image publish days later, not a red CI run.
Fix
- deps: pin past the yanked chacha20 0.10.1 #151 — merge this. It regenerates the lock (removes exactly the six orphans, qualifies the two bare
rand_corerefs asrand_core 0.9.5) and bumpschacha200.10.1 → 0.10.2, the yanked crate that is currently failingcargo-denyon every PR in this repo. Verified at its head91e8ec5: 0 ambiguous refs, 0 orphans. Green since 2026-09-07,REVIEW_REQUIRED. Merging it triggersrelease.docker.edge-connect.yml, which republishes all three variants against the current bases. - ci: build, lint and test with --locked #157 — add
--lockedto the three cargo invocations inrust.yml, so a lock Cargo would have to rewrite fails the PR that introduces it instead of a release. Merge after deps: pin past the yanked chacha20 0.10.1 #151.
Neither closes the underlying hole: two Dependabot PRs whose locks are each valid against their own base still merge cleanly into an invalid one. Requiring "branches must be up to date before merging" (or a merge queue) on main is the branch-protection setting that prevents it, and needs someone with repo admin.
Verification once #151 lands
docker buildx imagetools inspect ghcr.io/malbeclabs/doublezero:mainnet-beta-0.39.0-1 \
--format '{{ .Manifest.Digest }}'
docker buildx imagetools inspect ghcr.io/malbeclabs/doublezero-edge-connect:mainnet-beta \
--format '{{ index .Image.Config.Labels "org.opencontainers.image.base.digest" }}'
# the two digests must match
Activity
ci: build, lint and test with --locked
ci: build, lint and test with --locked
ci: build, lint and test with --locked (#157)
juan-malbeclabs commentedon Sep 18, 2026
Resolved by #151, which regenerated the lock file, and #157, which put
--lockedin front of build, lint and test so the lock and the manifest cannot drift apart again without a red check before the publish rather than inside it.Publishes have been green since:
release.docker.edge-connectonmainsucceeded on 2026-09-11 and again on 2026-09-18, and themainnet-betatag in GHCR was last pushed 2026-09-18 15:17 — so the image is no longer the 2026-09-07 build pinned to the 0.38.0 client base. Closing.