Skip to content

fix(devcontainer): replace flaky LLVM and kind features - #2826

Draft
Quang Nguyen (nddq) wants to merge 1 commit into
mainfrom
nddq/devcontainer-ci-flakes
Draft

Quang Nguyen (nddq) wants to merge 1 commit into
mainfrom
nddq/devcontainer-ci-flakes

Conversation

@nddq

Copy link
Copy Markdown
Member

Description

The DevContainer workflow fails often because two features download from hosts that GitHub runners cannot reach reliably.

  • LLVM: ghcr.io/devcontainers-community/features/llvm runs llvm.sh from apt.llvm.org and adds that apt repository. The requests to apt.llvm.org fail intermittently, and sometimes DNS returns only an IPv6 address that the build container cannot reach:

    [error] Distribution 'ubuntu' in version '24.04.5 LTS (Noble Numbat)' is not supported by this script.
    [error] GPG key not reachable at https://apt.llvm.org/llvm-snapshot.gpg.key
    Connecting to apt.llvm.org (apt.llvm.org)|2a04:4e42:79::561|:443... failed: Network is unreachable.
    

    The repository adds no packages. Its noble builds of LLVM 17 (1:17.0.6~++…) sort below Ubuntu's 1:17.0.6-9ubuntu1, so apt installs the Ubuntu packages anyway.

  • kind: ghcr.io/devcontainers-extra/features/kind resolves latest through the GitHub API without a token. Shared runner IPs hit the rate limit, and latest can resolve to an alpha tag that has no release:

    urllib.error.HTTPError: HTTP Error 403: rate limit exceeded
    nanolayer.installers.gh_release.resolvers.asset_resolver.AssetResolver.NoReleaseError: no release exists for repo:kubernetes-sigs/kind and tag: v0.33.0-alpha
    

The Go version check also passes when the versions differ. The Go feature installs 1.24.11, but go.mod requires go 1.26.0. With GOTOOLCHAIN=auto, go version downloads and reports go1.26.0.

This PR:

  • LLVM: drops the feature. The apt-deps step installs clang-17 and llvm-17 from the Ubuntu archive (the same packages as before) and creates the clang and llvm-strip symlinks.
  • kind: drops the feature. sigs.k8s.io/kind v0.33.0 joins the tool block in go.mod, and go-setup runs go install sigs.k8s.io/kind. go.mod marks tool dependencies // indirect, which Dependabot skips by default, so dependabot.yaml allows sigs.k8s.io/kind by name. Minor and patch updates arrive with the k8s group.
  • Go: sets the Go feature to 1.26.0 and runs the version check with GOTOOLCHAIN=local. Also pins golangciLintVersion to 2.11.4, the go.mod version that the golangci-lint workflow builds, so that the editor lints like CI.

Checklist

  • I have read the contributing documentation.
  • I signed and signed-off the commits (git commit -S -s ...). See this documentation on signing commits.
  • I have correctly attributed the author(s) of the code.
  • I have tested the changes locally.
  • I have followed the project's style guidelines.
  • I have updated the documentation, if necessary.
  • I have added tests, if applicable.

Screenshots (if applicable) or Testing Completed

Built the dev container with @devcontainers/cli 0.89.0 (the CLI version that devcontainers/ci uses) and ran the workflow's runCmd in it. All steps pass, including the Go version check:

Ubuntu clang version 17.0.6 (9ubuntu1)
Ubuntu LLVM version 17.0.6
go version go1.26.0 linux/amd64
kind v0.33.0 go1.26.0 linux/amd64

go generate rebuilds all 7 eBPF objects, and golangci-lint version reports 2.11.4. The module cache holds no downloaded Go toolchain. The DevContainer workflow runs on this PR because the PR changes .devcontainer/** and go.mod.

@github-actions

github-actions Bot commented Oct 3, 2026

Copy link
Copy Markdown

Retina Code Coverage Report

Total coverage no change

The DevContainer build fails intermittently in two features:

- The LLVM feature downloads llvm.sh from apt.llvm.org and adds that
  apt repository. Requests to apt.llvm.org from GitHub runners fail
  intermittently, and sometimes DNS returns only an IPv6 address that
  the build container cannot reach. The repository adds no packages.
  Its noble builds of LLVM 17 (1:17.0.6~++...) sort below Ubuntu's
  1:17.0.6-9ubuntu1, so apt installs the Ubuntu packages.
- The kind feature resolves "latest" through the GitHub API without a
  token. Shared runner IPs hit the rate limit (HTTP 403), and "latest"
  can resolve to an alpha tag that has no release (HTTP 404).

Install clang-17 and llvm-17 from the Ubuntu archive in the apt-deps
step in place of the LLVM feature. Add kind as a tool in go.mod, and
install it with go install, which downloads through the Go module
proxy. go.mod marks tool dependencies as indirect, and Dependabot
skips indirect dependencies by default, so allow sigs.k8s.io/kind by
name.

The Go version check also passes when the versions differ. The Go
feature installs 1.24.11, but go.mod requires 1.26.0. With
GOTOOLCHAIN=auto, go version downloads and reports the go.mod version.
Set the feature to 1.26.0, and run the check with GOTOOLCHAIN=local.
Pin golangci-lint to the go.mod tool version, which the lint workflow
builds, so that the editor lints like CI.

Signed-off-by: Quang Nguyen <28567936+nddq@users.noreply.github.com>
@nddq
Quang Nguyen (nddq) force-pushed the nddq/devcontainer-ci-flakes branch from 5a185a8 to d31b6f3 Compare October 6, 2026 15:00

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant