fix: build multi-platform images by digest and merge manifests#447
Conversation
7c1554f split each bake target's build into one job per platform (to use native ARM runners) but every job still pushed under the same final tags, so the three concurrent pushes for a target raced and only the last platform survived on Docker Hub/GHCR (#443). Each per-platform job now pushes its image by digest only (no tag), and a new merge job assembles the digests for a target into a single multi-arch manifest with docker buildx imagetools create, which also handles the cross-registry copy to both Docker Hub and GHCR. Cosign signing moves to the merge job so it signs the final manifest-list digest instead of a single-platform one. Tags are resolved once in the prepare job (docker buildx bake --print) and shared via an uploaded artifact, since docker-bake.hcl embeds a timestamp in versioned tags that would otherwise diverge if recomputed independently by each job.
|
Warning Review limit reached
Next review available in: 41 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (1)
📝 WalkthroughWalkthroughThe release publishing workflow was restructured into a digest-based multi-arch flow: prepare generates bake metadata and a build matrix, build pushes each platform image by digest and uploads digests as artifacts, and merge assembles a signed multi-arch manifest list from those digests. ChangesPublish Workflow Digest-Based Multi-Arch Build
Estimated code review effort: 4 (Complex) | ~45 minutes Sequence Diagram(s)sequenceDiagram
participant Prepare
participant Build
participant Merge
participant Registry
Prepare->>Prepare: docker buildx bake --print default
Prepare->>Registry: upload bake-metadata.json artifact
Build->>Registry: download bake-metadata.json
Build->>Registry: push platform image by digest
Build->>Registry: upload per-target digest artifacts
Merge->>Registry: download digest artifacts
Merge->>Registry: docker buildx imagetools create (manifest list)
Merge->>Registry: cosign sign manifest digest
Related issues: Suggested labels: ci, workflow, docker Suggested reviewers: dune73, theseion 🐰 A rabbit bakes each arch by digest deep, 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Co-authored-by: Max Leske <250711+theseion@users.noreply.github.com>
Co-authored-by: Max Leske <250711+theseion@users.noreply.github.com>
Co-authored-by: Max Leske <250711+theseion@users.noreply.github.com>
Co-authored-by: Max Leske <250711+theseion@users.noreply.github.com>
Co-authored-by: Max Leske <250711+theseion@users.noreply.github.com>
Summary
Fixes #443 without reverting the native-ARM-runner speedup from 7c1554f (see #445 for the revert alternative).
push-by-digest=true, no tag) to one registry. A newmergejob (one per target) downloads all of that target's platform digests and assembles them into a single multi-arch manifest withdocker buildx imagetools create, pushing to both Docker Hub and GHCR in the same call (imagetools createcopies cross-registry as needed).mergejob so it signs the final manifest-list digest, not a single-platform one.docker-bake.hcl's versioned tags embed a timestamp (formatdate("YYYYMMDDHHMM", timestamp())), so tags are resolved once inprepareviadocker buildx bake --printand shared to every job through an uploaded artifact — otherwise each job re-evaluating the bake file would compute different timestamps and the merge would target mismatched tags.Validation
Since this touches the production release pipeline, I validated the core mechanics locally against two throwaway local registries (not in CI) before opening this:
docker buildx bake --printwith--set target.tags=+--set target.output=type=image,name=...,push-by-digest=true,...resolves cleanly (tags fully cleared, single custom output).linux/amd64andlinux/arm64by digest against a local registry, then randocker buildx imagetools createwith two-tdestinations across two separate local registries. Confirmed: the merge correctly flattened the per-platform attestation manifests into one clean multi-arch index, and the cross-registry copy to the registry with zero prior content succeeded, producing an identical manifest in both.actionlintandzizmorboth pass clean on the rewritten workflow (no template-injection or credential-persistence findings).Test plan
docker buildx imagetools inspect <tag>shows all 3 platforms) on both Docker Hub and GHCRAI Disclosure
Per the project's AI-Assisted Contribution Policy:
.github/workflows/publish.yml(theprepare/build/mergejob split, freezing bake tags once viadocker buildx bake --printto avoid timestamp drift, and thejq/imagetools createorchestration).actionlintandzizmorrun against the workflow (zero findings). Before opening this PR, the risky part of the design — clearing a bake target'stagsin favor of a digest-onlyoutput, and cross-registry manifest assembly viadocker buildx imagetools create— was validated end-to-end against two throwaway local Docker registries (not just reasoned about), confirming the merge correctly flattens per-platform provenance/attestation manifests and replicates cross-registry. I reviewed the final workflow logic and the local validation results myself before publishing.Summary by CodeRabbit