ci: Build and publish multi-arch (x86-64 and ARM64) container images and also run Rust tests on ARM64 runners. - #428
Open
jackluo923 wants to merge 2 commits into
Open
Conversation
Contributor
WalkthroughThe image workflow builds services on native amd64 and arm64 runners, then publishes multi-architecture manifests for non-pull-request events. The huntsman test matrix now includes Ubuntu ARM runners. ChangesNative multi-architecture CI
Estimated code review effort: 4 (Complex) | ~45 minutes Sequence Diagram(s)sequenceDiagram
participant GitHubActions
participant NativeRunners
participant ArtifactStorage
participant GHCR
participant Buildx
GitHubActions->>NativeRunners: Build each service for amd64 and arm64
NativeRunners->>ArtifactStorage: Upload architecture-specific digests
GitHubActions->>ArtifactStorage: Download digests
GitHubActions->>GHCR: Authenticate
GitHubActions->>Buildx: Create tagged multi-architecture manifests
Buildx->>GHCR: Publish manifests
GitHubActions->>GHCR: Inspect the resulting manifest
Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
jackluo923
force-pushed
the
ci/multi-arch-images
branch
from
August 6, 2026 07:28
245ddd0 to
7e5483c
Compare
…d arm64 test legs. Builds each service on a native runner per arch, pushes each arch by digest, and merges the per-arch digests into one multi-arch manifest per service.
jackluo923
force-pushed
the
ci/multi-arch-images
branch
from
August 6, 2026 07:29
7e5483c to
1f7aa1e
Compare
jackluo923
marked this pull request as ready for review
August 6, 2026 07:34
LinZhihao-723
requested changes
Aug 6, 2026
…or `huntsman-tests`. The C++ implementation is unofficially deprecated and slated for removal, so there's no value in adding ARM64 coverage for it. `wolf-tests` is restored to match `main` exactly, which also makes the `rm --force` workaround unnecessary since the job no longer runs on ARM64 runners. `huntsman-tests` keeps its ARM64 legs: the Rust suite builds and runs natively on ARM64 with no changes to the dependency install scripts.
jackluo923
changed the title
jackluo923
changed the title
jackluo923
changed the title
jackluo923
force-pushed
the
ci/multi-arch-images
branch
from
August 6, 2026 16:55
0c4fd4e to
1cfb0a3
Compare
Contributor
|
Note GitHub couldn't provide a complete incremental comparison for this pull request, so CodeRabbit is performing a full review instead. This review may take a little longer. |
junhaoliao
approved these changes
Aug 12, 2026
Comment on lines
+42
to
+43
| # Each arch is built on a native runner rather than through QEMU emulation, since | ||
| # emulating the Rust build would be an order of magnitude slower. |
Member
There was a problem hiding this comment.
this design decision may not need to live in the workflow; i think we can just document only what the matrix mapping does
Suggested change
| # Each arch is built on a native runner rather than through QEMU emulation, since | |
| # emulating the Rust build would be an order of magnitude slower. | |
| # Map each architecture to its native runner. |
|
|
||
| - name: "Upload digest" | ||
| if: "github.event_name != 'pull_request'" | ||
| uses: "actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f" # v6.0.0 |
Member
There was a problem hiding this comment.
Suggested change
| uses: "actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f" # v6.0.0 | |
| uses: "actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a" # v7.0.1 |
| echo "name=${GITHUB_REPOSITORY,,}" >> "$GITHUB_OUTPUT" | ||
|
|
||
| - name: "Download digests" | ||
| uses: "actions/download-artifact@018cc2cf5baa6db3ef3c5f8a56943fffe632ef53" # v6.0.0 |
Member
There was a problem hiding this comment.
Suggested change
| uses: "actions/download-artifact@018cc2cf5baa6db3ef3c5f8a56943fffe632ef53" # v6.0.0 | |
| uses: "actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c" # v8.0.1 |
| pattern: "digests-${{matrix.service}}-*" | ||
| merge-multiple: true | ||
|
|
||
| - uses: "docker/setup-buildx-action@b5ca514318bd6ebac0fb2aedd5d36ec1b5c232a2" # v3.10.0 |
Member
There was a problem hiding this comment.
Suggested change
| - uses: "docker/setup-buildx-action@b5ca514318bd6ebac0fb2aedd5d36ec1b5c232a2" # v3.10.0 | |
| - uses: "docker/setup-buildx-action@bb05f3f5519dd87d3ba754cc423b652a5edd6d2c" # v4.2.0 |
|
|
||
| - uses: "docker/setup-buildx-action@b5ca514318bd6ebac0fb2aedd5d36ec1b5c232a2" # v3.10.0 | ||
|
|
||
| - uses: "docker/login-action@4907a6ddec9925e35a0a9e82d7399ccc52663121" # v4.1.0 |
Member
There was a problem hiding this comment.
Suggested change
| - uses: "docker/login-action@4907a6ddec9925e35a0a9e82d7399ccc52663121" # v4.1.0 | |
| - uses: "docker/login-action@dbcb813823bdd20940b903addbd779551569679f" # v4.6.0 |
| username: "${{github.actor}}" | ||
| password: "${{secrets.GITHUB_TOKEN}}" | ||
|
|
||
| - uses: "docker/metadata-action@902fa8ec7d6ecbf8d84d538b9b233a880e428804" # v5.7.0 |
Member
There was a problem hiding this comment.
Suggested change
| - uses: "docker/metadata-action@902fa8ec7d6ecbf8d84d538b9b233a880e428804" # v5.7.0 | |
| - uses: "docker/metadata-action@dc802804100637a589fabce1cb79ff13a1411302" # v6.2.0 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
The published container images (
storage,scheduler,worker) are x86-64 only, so they can't run on ARM64 hosts. This publishes multi-arch manifests instead, and adds ARM64 legs tohuntsman-testsso that ARM64 breakage in the Rust implementation surfaces in review instead of after publishing.spider-image-build:ubuntu-24.04/ubuntu-24.04-arm) rather than under QEMU, since emulating the Rust build would be roughly an order of magnitude slower.merge-manifestsjob combines the per-arch digests into one multi-arch manifest per service.merge-manifeststhen stamps every tag thatdocker/metadata-actiongenerates (branch, tag, sha) onto the one index, so no tag has to be enumerated in the merge step and no arch-suffixed tags reach the registry.provenanceis disabled so that each arch is pushed as a plain image manifest. Buildx otherwise wraps each build in a single-entry manifest list to carry the attestation, anddocker buildx imagetools createcannot nest one index inside another.type=cacheonly) and push nothing.tests:huntsman-testsalso runs onubuntu-22.04-armandubuntu-24.04-arm.wolf-testsis left unchanged. The C++ implementation is unofficially deprecated and will be removed soon, so ARM64 coverage for it isn't worth adding.Checklist
Validation performed
Both workflows were run on a fork (
jackluo923/spider):spider-image-build: all 6 build legs and all 3merge-manifestslegs passed. TheInspect multi-arch manifeststep confirms that each service resolves to an OCI image index withlinux/amd64andlinux/arm64children.tests: the ARM64 legs ofhuntsman-testspassed, so the Rust suite builds and runs natively on ARM64 with no changes to the dependency install scripts. The run also covered ARM64 legs ofwolf-teststhat this PR no longer adds.yamllint --strict(using the repo's shared config) andactionlintare clean on both files.