Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/requirements-ci.txt
Original file line number Diff line number Diff line change
@@ -1 +1 @@
pre-commit==4.6.1
pre-commit==4.6.2
5 changes: 4 additions & 1 deletion .github/workflows/pr-validation.yml
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,9 @@ jobs:
- name: Checkout
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7

- name: Check testsuite workflow contract
run: python3 scripts/check-testsuite-workflow-ref.py

- name: Check for undeclared gitlinks
shell: bash
env:
Expand Down Expand Up @@ -228,7 +231,7 @@ jobs:
permissions:
contents: read
packages: write # testsuite pushes screenshot OCI artifacts to GHCR
# Keep the testsuite SHA centralized in run-testsuite.yml.
# Keep the testsuite workflow ref centralized in run-testsuite.yml.
uses: ./.github/workflows/run-testsuite.yml
with:
image: ghcr.io/projectbluefin/bluefin:testing
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/scorecard.yml
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,6 @@ jobs:
# Upload the results to GitHub's code scanning dashboard (optional).
# Commenting out will disable upload of results to your repo's Code Scanning dashboard
- name: "Upload to code-scanning"
uses: github/codeql-action/upload-sarif@5595ccaf912efad79be6eef63a5619ff05969be3 # v4
uses: github/codeql-action/upload-sarif@ff2f1c621b7f889edc0d3c761ac2e6a3f8cdb0dd # v4
with:
sarif_file: results.sarif
5 changes: 5 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,11 @@ devcontainer
# Scratch checkouts of sibling projectbluefin repos
.common-ref/

# Per-test scratch roots created by the bats unit suite. Tests that abort
# leave theirs behind, and an unsuspecting `git add -A` commits hundreds of
# stub files.
tests/unit/.bats-sandbox/

# Python cache
__pycache__/
*.py[cod]
Expand Down
23 changes: 23 additions & 0 deletions build_files/base/20-tests.sh
Original file line number Diff line number Diff line change
Expand Up @@ -24,13 +24,36 @@ done

test -f /usr/share/ublue-os/homebrew/fonts.Brewfile

# bluefinctl and the default CLI set (fzf, starship, htop, ...) are not baked
# into the image — they are installed per-user at first graphical login by the
# brew-preinstall user service. The whole delivery path is inherited from the
# pinned `common` image, whose digest Renovate bumps automatically, so a rename
# or drop upstream would silently stop shipping bluefinctl with no other signal.
# Assert the pieces the service actually needs: the Brewfiles it reads, the
# binary its ExecStart points at, the unit, and the preset that enables it.
# See: https://github.com/projectbluefin/bluefin/issues/965
test -f /usr/share/ublue-os/homebrew/preinstall.d/bluefinctl.Brewfile
test -f /usr/share/ublue-os/homebrew/preinstall.d/system-cli.Brewfile
test -x /usr/bin/brew-preinstall
test -f /usr/lib/systemd/user/brew-preinstall.service
grep -q '^enable brew-preinstall\.service$' /usr/lib/systemd/user-preset/01-brew-preinstall.preset

# If this file is not on the image bazaar will automatically be removed from users systems :(
# See: https://docs.flatpak.org/en/latest/flatpak-command-reference.html#flatpak-preinstall
test -f /usr/share/flatpak/preinstall.d/bazaar.preinstall

# Make sure this garbage never makes it to an image
test -f /usr/lib/systemd/system/flatpak-add-fedora-repos.service && false

# Framework laptops need this modprobe option to get battery charge limiting.
# The in-tree cros_charge-control driver refuses to bind when the EC advertises
# Framework's own charge control, so without the option there is no
# charge_control_end_threshold sysfs node and charge limiting silently
# disappears on Framework hardware.
# See: https://github.com/projectbluefin/bluefin/issues/879
test -f /usr/lib/modprobe.d/fw-charge-control.conf
grep -q '^options cros_charge_control probe_with_fwk_charge_control=1$' /usr/lib/modprobe.d/fw-charge-control.conf

IMPORTANT_PACKAGES=(
anaconda-live
distrobox
Expand Down
12 changes: 6 additions & 6 deletions docs/skills/ci/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -127,12 +127,12 @@ collects coverage but does not own pass/fail.
- Preserve action pinning and workflow permissions.
- Do not add PAT-based authentication.
- Keep end-to-end suites on their configured event.
- Reference `projectbluefin/testsuite`'s reusable E2E workflow through its
managed `@v1` tag, never an immutable digest; testsuite advances `v1` after
each successful main-branch merge. `config:best-practices` pins action refs
to digests, so `projectbluefin/testsuite` is disabled for the
`github-actions` manager in `.github/renovate.json5`; without that rule
Renovate re-pins the ref and freezes the gate on a stale test tree.
- Reference `projectbluefin/testsuite`'s reusable E2E workflow by its managed
`@v1` tag, never a digest; testsuite advances `v1` after each successful main
merge. It is disabled for the `github-actions` Renovate manager in
`.github/renovate.json5`, or re-pinning freezes the gate on a stale test tree.
- Route testsuite calls through `.github/workflows/run-testsuite.yml`;
`scripts/check-testsuite-workflow-ref.py` enforces that and `test_ref: v1`.
- Update this skill when workflow behavior changes.

## Verification
Expand Down
24 changes: 24 additions & 0 deletions docs/skills/ci/references/workflow-map.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,3 +9,27 @@ For current workflows:
find .github/workflows -maxdepth 1 -type f -name '*.yml' -o -name '*.yaml' | sort
git grep -n '^name:\|^on:' .github/workflows
```

## The testsuite reference contract

`.github/workflows/run-testsuite.yml` is the only workflow that may reference
`projectbluefin/testsuite/.github/workflows/e2e.yml` directly. Every other
caller goes through that wrapper, so the ref and `test_ref` are set in exactly
one place.

Two invariants, both enforced by `scripts/check-testsuite-workflow-ref.py` in
the `validate` job:

- the reference is `@v1` — testsuite advances that tag after each successful
main-branch merge, so a digest pin silently freezes the gate on a stale test
tree (this is the #929 regression);
- the wrapper passes `test_ref: v1`.

Renovate would otherwise undo the first one: `config:best-practices` pins
action refs to digests, so `projectbluefin/testsuite` is excluded from the
`github-actions` manager in `.github/renovate.json5`. Removing that exclusion
re-pins the ref and reintroduces the same freeze.

```bash
python3 scripts/check-testsuite-workflow-ref.py
```
2 changes: 1 addition & 1 deletion docs/skills/security/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ do not invent a replacement key or trust path.

- [COPR isolation invariant](references/copr-isolation.md)
- [signing and verification](references/signing.md)
- [CVE-2026-33186 grpc in buildah](references/cve-2026-33186-grpc-buildah.md)
- [CVE-2026-33186 grpc in buildah and podman](references/cve-2026-33186-grpc-buildah.md)

## When to Use

Expand Down
134 changes: 101 additions & 33 deletions docs/skills/security/references/cve-2026-33186-grpc-buildah.md
Original file line number Diff line number Diff line change
@@ -1,50 +1,118 @@
# CVE-2026-33186 (google.golang.org/grpc) in the buildah binary
# CVE-2026-33186 (google.golang.org/grpc) in the Fedora container-tools stack

Severity: **Critical** (CVSS ≥ 9.0, per the vulnerability scan gate).

## Finding

CVE-2026-33186 affects `google.golang.org/grpc v1.72.2`, which is bundled in the
`buildah` binary shipped by the Fedora container-tools stack. There is **no
self-contained fix** that this repository can apply on the Fedora 44 target.
CVE-2026-33186 affects `google.golang.org/grpc` before `v1.79.3`, which is
statically bundled into Go binaries shipped by the Fedora container-tools stack.
There is **no self-contained fix** that this repository can apply on the Fedora
44 target.

## Root cause

- The vulnerable module is bundled in `buildah`, not in an explicitly declared
image package. `buildah` is not listed in `build_files/packages/base.toml`; it
arrives as part of the standard Fedora Silverblue container-tools set
(podman/buildah/skopeo) and is not excluded.
- `containers/buildah@v1.43.2/go.mod` pins `google.golang.org/grpc v1.72.2 // indirect`
(the vulnerable version).
- Fedora 44's latest available buildah is `buildah-1.43.2-1.fc44` (stable since
`2026-06-19`), which still bundles grpc v1.72.2.
- As of the last check (`2026-08-07`), Bodhi lists **no** F44 `buildah` update in
`testing` or `pending` status. There is no newer F44 RPM to pin or override to.
- Fedora 45 carries a fixed buildah: `buildah-1.45.0-2.fc45` bundles
`google.golang.org/grpc v1.82.1`. The fix is therefore gated on a Fedora 45
migration, not on an in-repo pin.
The vulnerable module is bundled in the binaries, not in an explicitly declared
image package. None of these are listed in `build_files/packages/base.toml`; they
arrive as part of the standard Fedora Silverblue container-tools set and are not
excluded.

**Two carriers ship the vulnerable version on Fedora 44, not one:**

| F44 package | grpc pinned in that version | Status |
|---|---|---|
| `buildah-2:1.43.2-1.fc44` | `v1.72.2` | vulnerable |
| `podman-5.8.4-1.fc44` | `v1.72.2` | vulnerable |
| `skopeo-1.22.2-2.fc44` | `v1.79.3` | already fixed |

`skopeo` is listed to show the finding is per-binary: the same stack already
carries a fixed build, so "container-tools is vulnerable" is too coarse. Fixing
one carrier does not clear the scan while the other remains.

## Why an in-repo pin/override does not resolve it

The `Pins and Overrides` section in `build_files/base/03-packages.sh` documents the
mechanism for `rpm-ostree override replace`, but it requires a fixed RPM that
exists in a Fedora repo. No such F44 buildah RPM exists. Downgrading is not
helpful (older versions are also vulnerable), and no F44 `buildah` build bumps
grpc to `>= 1.79.3`.
The `Pins and Overrides` section in `build_files/base/03-packages.sh` documents
the mechanism for `rpm-ostree override replace`, but it requires a fixed RPM that
exists in a Fedora repo. No such F44 RPM exists for either carrier. Downgrading
does not help — older versions are also vulnerable.

## Where the upstream fixes land

| Project | First fixed release | grpc after fix |
|---|---|---|
| `buildah` | `v1.44.0` | `v1.81.1` |
| `podman` | `v6.0.2` | `v1.81.1` |

**The podman side is the blocker.** buildah's fix is a minor bump that Fedora
could reasonably carry, but podman's fix first appears in `v6.0.2` — the latest
`5.8.x` release, `v5.8.5`, still pins `grpc v1.72.2`. There is no podman 5.x
release that resolves this, so clearing it on F44 would require Fedora either to
rebase podman from 5.8.x to 6.x inside a stable release (unlikely) or to carry a
backported grpc bump.

Consequently: **an F44 `buildah` update alone will not clear this finding.** The
realistic fix is the Fedora 45 migration, which moves both carriers to fixed
versions at once — not a secondary note, but the primary path.

## Action

Wait for/track an upstream Fedora 44 `buildah` update (>= v1.44.0, or a 1.43.x
backport that bumps grpc to >= 1.79.3). Once that RPM reaches the F44 repos, the
next `testing` build picks it up automatically. The durable fix is the Fedora 45
migration. Do not weaken the vulnerability scan gate or add an exception; this
finding is a tracking note, not an approval to suppress the scan.
Track the Fedora 45 migration as the fix. An F44 `buildah` update is worth
picking up when it appears but should not be expected to close the alert on its
own; re-scan and confirm rather than assuming. Do not weaken the vulnerability
scan gate or add an exception; this finding is a tracking note, not an approval
to suppress the scan.

## A new issue number is not a new problem

`bootc-build/scan-image` files a fresh `fix(security): critical CVE detected …`
issue for non-PR builds, and its duplicate check keys on the ref, so a `main`
build does not match an existing `testing` issue. This CVE has been filed at
least four times (#530, #638, #870, #919) — roughly monthly, not per build.
Before triaging one of these as new, compare the CVE ID against this file. The
dedup logic lives in `projectbluefin/actions`, not here.

## Re-checking this (do not re-derive by hand)

This finding has been re-triaged several times, each time by repeating the same
lookups. Both facts are one command each.

Current F44 versions of the carriers:

```bash
for p in buildah podman skopeo; do
printf '%-8s ' "$p"
curl -s "https://mdapi.fedoraproject.org/f44/pkg/$p" | jq -r '"\(.version)-\(.release)"'
done
# buildah 1.43.2-1.fc44
# podman 5.8.4-1.fc44
# skopeo 1.22.2-2.fc44
```

Parse the JSON rather than grepping it — an mdapi response embeds a `"version"`
field for every bundled Go module, so a bare `grep` returns hundreds of lines
and the RPM's own version is easy to mistake for one of them.

The grpc version any given upstream release bundles:

```bash
curl -s https://raw.githubusercontent.com/containers/podman/v5.8.4/go.mod |
grep google.golang.org/grpc
```

The finding clears when **every** carrier above resolves to a version whose
`go.mod` pins `grpc >= v1.79.3`.

## Verification

- Fedora 44 updates repo: `buildah-1.43.2-1.fc44` (no newer version).
- Bodhi F44 buildah: no `testing` or `pending` updates as of `2026-08-07`.
- `containers/buildah@v1.43.2/go.mod` → `google.golang.org/grpc v1.72.2 // indirect`.
- `containers/buildah@v1.45.0/go.mod` → `google.golang.org/grpc v1.82.1`.
- Bluefin `Containerfile` target: `ARG FEDORA_MAJOR_VERSION="44"`; the latest
successful testing build resolved `fedora_version=44`.
Verified `2026-08-10` (F44 versions via `mdapi.fedoraproject.org`, grpc pins via
each project's `go.mod` at the exact shipped tag):

- `f44/buildah` → `1.43.2-1.fc44`; `containers/buildah@v1.43.2` → `grpc v1.72.2`.
- `f44/podman` → `5.8.4-1.fc44`; `containers/podman@v5.8.4` → `grpc v1.72.2`.
- `f44/skopeo` → `1.22.2-2.fc44`; `containers/skopeo@v1.22.2` → `grpc v1.79.3`.
- `containers/buildah@v1.44.0` → `grpc v1.81.1` (first fixed buildah).
- `containers/podman@v5.8.5` → `grpc v1.72.2`; `@v6.0.2` → `grpc v1.81.1`.
- Bluefin `Containerfile` target: `ARG FEDORA_MAJOR_VERSION="44"`.

Not verified here: which carrier a specific Trivy alert instance points at.
Reading code-scanning alert locations needs permissions this check did not have,
and it does not change the conclusion — both carriers must be fixed regardless.
4 changes: 2 additions & 2 deletions image-versions.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,8 @@ images:
- name: common
image: ghcr.io/projectbluefin/common
tag: latest
digest: sha256:feeeb1d93aafdd1ba250abbf93da3f8416e648228ceaf75531a808b3d5abd94a
digest: sha256:fb943c87866292fb74eb74610e9cd08a1a91fe42e763e28473f3f57cf18f26a5
- name: brew
image: ghcr.io/ublue-os/brew
tag: latest
digest: sha256:de0391c67209703bdf1249079c8d478d44eff864d62e7ec6f12aaa382bdf21df
digest: sha256:8f952ae54585db9f855a306ef365e13609ed7c7944b12b823ba7d5ce8e1a145b
57 changes: 57 additions & 0 deletions scripts/check-testsuite-workflow-ref.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
#!/usr/bin/env python3
"""Enforce the Bluefin-to-testsuite reusable-workflow contract."""

from pathlib import Path
import re
import sys


ROOT = Path(__file__).resolve().parents[1]
WORKFLOW_DIR = ROOT / ".github" / "workflows"
WRAPPER = WORKFLOW_DIR / "run-testsuite.yml"
WORKFLOW_REF = re.compile(
r"^\s+uses:\s+projectbluefin/testsuite/\.github/workflows/e2e\.yml@([^\s#]+)",
re.MULTILINE,
)
TEST_REF = re.compile(r"^\s+test_ref:\s+([^\s#]+)", re.MULTILINE)


def main() -> int:
errors: list[str] = []
wrapper_refs = WORKFLOW_REF.findall(WRAPPER.read_text(encoding="utf-8"))

if wrapper_refs != ["v1"]:
errors.append(
f"{WRAPPER.relative_to(ROOT)} must contain exactly one direct testsuite "
f"workflow reference at @v1; found {wrapper_refs!r}"
)

wrapper_test_refs = TEST_REF.findall(WRAPPER.read_text(encoding="utf-8"))
if wrapper_test_refs != ["v1"]:
errors.append(
f"{WRAPPER.relative_to(ROOT)} must pass exactly one test_ref: v1; "
f"found {wrapper_test_refs!r}"
)

for workflow in sorted(WORKFLOW_DIR.glob("*.y*ml")):
if workflow == WRAPPER:
continue
refs = WORKFLOW_REF.findall(workflow.read_text(encoding="utf-8"))
if refs:
errors.append(
f"{workflow.relative_to(ROOT)} calls testsuite e2e directly; "
"call the local run-testsuite wrapper instead"
)

if errors:
print("Testsuite workflow contract failed:", file=sys.stderr)
for error in errors:
print(f"- {error}", file=sys.stderr)
return 1

print("Testsuite workflow contract passed: canonical wrapper uses @v1 with test_ref: v1")
return 0


if __name__ == "__main__":
raise SystemExit(main())
Loading