feat: verify reviewed managed image digests by Nickfost · Pull Request #74 · RandomDevelopment/ci-fleet

Nickfost · 2026-08-16T02:45:44Z

Summary

pin multi-architecture Debian and Go bases to official registry manifest-list digests
fix Debian package inputs to a signed snapshot and Docker CLI/Compose artifacts to exact upstream versions and SHA-256 values
require reviewed controller/runner image IDs per engine commit and host architecture
verify exact image IDs during drift, convergence, missing-image recovery, and managed rollback
capture exact engine/image IDs before upgrading a pre-digest installation and fail closed on unverifiable historical checkpoints
commit and consume the controller Go module checksum graph without regenerating dependency metadata during image builds

Validation

scripts/test-install-worker-controller.sh
python3 scripts/test_desired_state.py
python3 templates/config-repository/scripts/test_policy.py
templates/config-repository/scripts/validate.sh
Go tests and go mod verify
shell syntax, docs link validation, and committed-secret scan
official registry/package URLs and all four downloaded Docker package artifacts matched the recorded SHA-256 values

Remaining gate

Draft: no managed-image IDs were invented. A clean BuildKit build must be repeated for each selected architecture from the immutable engine commit, each matching local image-ID pair must be reviewed and recorded in the authorized private configuration, and the exact config commit must pass its separate review/evidence gate before this can merge. This change does not mutate that external repository or any live Docker host.

Closes #36 only after that external evidence exists.

Nickfost · 2026-08-16T02:46:09Z

@codex review

Focused exact-head review at 30386ff. Verify immutable upstream inputs, per-engine reviewed image-ID selection, convergence and rollback exact-ID enforcement, tamper/missing-image regressions, and safe legacy-checkpoint handling. Apply the objective review-value gate; collapse duplicate/speculative variants.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 30386ff273

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Nickfost · 2026-08-16T03:25:45Z

@codex review

Exact-head confirmation at 6517672. The only follow-up supplies the newly required reviewed image-ID arguments to CI initializer smoke tests after run 31922646396 exposed the stale valid-path invocation. Review the complete current head.

chatgpt-codex-connector · 2026-08-16T03:29:54Z

Codex Review: Didn't find any major issues. Keep them coming!

Reviewed commit: 65176720af

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Nickfost · 2026-08-16T06:35:51Z

@codex review

Exact-head review at b3386e1. Verify architecture-specific reviewed image-ID selection, pre-digest checkpoint capture/fail-closed rollback, and committed/consumed Go module checksums. Apply the objective review-value gate and avoid duplicate or speculative variants.

chatgpt-codex-connector · 2026-08-16T06:48:10Z

Codex Review: Didn't find any major issues. Hooray!

Reviewed commit: b3386e19b7

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Nickfost · 2026-08-17T03:46:47Z

@codex review

[lane-b:pr74:review:42611bc0] Exact-head review requested for 42611bc. Focus on the reproduced clean-build defect: BuildKit ignored the Dockerfile ARG default for exporter timestamp rewriting, so two no-cache AMD64/ARM64 OCI builds produced different configs, layers, manifests, and indexes. Verify that the Compose path now passes SOURCE_DATE_EPOCH explicitly to both managed builds and that the focused contract check prevents regression without weakening immutable input or rollback enforcement.

chatgpt-codex-connector · 2026-08-17T04:01:43Z

Codex Review: Didn't find any major issues. Breezy!

Reviewed commit: 42611bc082

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Nickfost · 2026-08-17T05:25:07Z

@codex review

[lane-b:pr74:review:376a13c3] Exact-head review requested for 376a13c. Focus on deterministic managed-image construction: timestamp propagation, apt-log cleanup, filesystem timestamp normalization, and the read-only runtime bind exclusions.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 376a13c3f3

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Nickfost · 2026-08-17T06:39:21Z

@codex review

[lane-b:pr74:review:489eca1d] Exact-head review requested for 489eca1. Focus on deterministic managed-image construction, especially installing the controller binary inside the same timestamp-normalized runtime layer via the read-only BuildKit stage mount.

chatgpt-codex-connector · 2026-08-17T06:54:55Z

Codex Review: Didn't find any major issues. Breezy!

Reviewed commit: 489eca1dbd

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Nickfost · 2026-08-17T09:10:15Z

@codex review

[lane-b:pr74:review:427fa206] Exact-head review requested for 427fa20. Focus on deterministic managed-image construction, including removal of ldconfig auxiliary cache content after both runner package-install phases.

chatgpt-codex-connector · 2026-08-17T09:15:40Z

Codex Review: Didn't find any major issues. What shall we delve into next?

Reviewed commit: 427fa2067c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Nickfost · 2026-08-17T11:52:28Z

@codex review

[lane-b:pr74:review:942f0307] Exact-head review requested for 942f030 after closing findings 3794020411, 3794020415, 3794020416, and 3794020422. Focus on deterministic managed-image construction and complete rebuild inputs.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 942f0307d0

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Nickfost · 2026-08-17T14:51:33Z

@codex review

[lane-b:pr74:review:9ba44a60] Exact-head review requested for 9ba44a6 after closing findings 3796175992, 3796176000, 3796176006, and 3796176011. Focus on two-step managed-image rollout safety, strict ID validation, and reproducible-build prerequisites.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 9ba44a6068

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Nickfost · 2026-08-17T15:32:57Z

@codex review

[lane-b:pr74:review:7ca616fb] Exact-head review requested for 7ca616f after closing findings 3797015820 and 3797015821. Focus on manager-only staging trust boundaries and exact pinned BuildKit rebuild behavior.

chatgpt-codex-connector · 2026-08-17T15:43:25Z

💡 Codex Review

ci-fleet/scripts/install-worker-controller.sh

Line 364 in 7ca616f

--build-arg CI_FLEET_COMMIT --build-arg SOURCE_DATE_EPOCH \

Set the reviewed build timestamp explicitly

When the installer is launched from an environment that exports SOURCE_DATE_EPOCH—for example, an operator shell or reproducible-build wrapper—the value-less --build-arg SOURCE_DATE_EPOCH forwards that ambient value instead of the reviewed 1786752000 used in docs/MANAGED-IMAGES.md:41-46. Because the rendered environment sourced above does not define this variable, the resulting layers can differ from the reviewed image IDs, making convergence and rollback fail their exact-ID checks; pass the reviewed value explicitly or clear the caller environment.

AGENTS.md reference: AGENTS.md:L39-L39

ci-fleet/templates/config-repository/scripts/validate.py

Lines 529 to 530 in 7ca616f

    
               changed_engine = changed_engine or old.get("engine_ref") != new.get("engine_ref") 
        
           validation.require(changed_engine, "$.controllers", "manager-only staging requires an engine_ref change")

Require every controller engine to advance during staging

In a multi-controller configuration, this fleet-wide changed_engine flag lets validation succeed when only one controller changes engine_ref. Any unchanged controller then receives the same fieldless staging revision, but its installer rejects it at scripts/install-worker-controller.sh:1135-1137 because the rendered and desired engine refs are equal, so that target can never report MANAGER_STAGED and the documented second rollout step is blocked. Validate an engine-ref change for each controller that must consume the later managed_images revision.

ci-fleet/scripts/install-worker-controller.sh

Lines 1105 to 1106 in 7ca616f

    
           [[ $(current_runtime_release) != "$release_dir" ]] || die 'manager-only staging activated an unreviewed runtime' 
        
           note "CHECK_STAGED manager_ref=$engine_ref runtime_unchanged=true"

Verify the old runtime before reporting staged

During manager-only staging, this inequality proves only that the current runtime symlink does not point at the newly staged release; it does not prove that the old release is still selected or that its controller is running with the installed image and environment. If the current link drifts to another release or the controller stops after staging, --check still returns success with runtime_unchanged=true, so an operator can proceed to the digest-bearing revision despite a broken target. Compare the current runtime and live controller against the installed rendered state before reporting CHECK_STAGED.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

feat: verify reviewed managed image digests

30386ff

chatgpt-codex-connector Bot reviewed Aug 16, 2026

View reviewed changes

Comment thread templates/config-repository/fleet.schema.json

Comment thread scripts/install-worker-controller.sh

Comment thread docs/MANAGED-IMAGES.md

test: supply managed image digests in CI

6517672

Nicks Hermes added 2 commits August 16, 2026 01:05

fix: bind managed images to architecture

94ec00e

fix: checkpoint legacy engine revision

b3386e1

Nickfost mentioned this pull request Aug 16, 2026

Make managed image rebuilds reproducible for pinned engines #36

Open

Nicks Hermes added 2 commits August 16, 2026 22:40

fix: pass reproducible timestamp to managed builds

fd2573f

test: keep reproducibility assertion shellcheck-clean

42611bc

Nicks Hermes added 3 commits August 17, 2026 00:11

fix: normalize managed image filesystem timestamps

1af7b2e

fix: avoid immutable build mounts during normalization

84d2c70

fix: exclude runtime bind files from normalization

376a13c

chatgpt-codex-connector Bot reviewed Aug 17, 2026

View reviewed changes

Comment thread runner/Dockerfile

Comment thread docs/MANAGED-IMAGES.md

Comment thread runner/Dockerfile Outdated

Comment thread runner/Dockerfile Outdated

fix: normalize controller copy in one runtime layer

489eca1

fix: remove nondeterministic runner linker cache

427fa20

Nicks Hermes added 2 commits August 17, 2026 06:43

fix: pin remaining managed image build inputs

8664d64

test: keep deterministic shadow assertion literal

942f030

chatgpt-codex-connector Bot reviewed Aug 17, 2026

View reviewed changes

Comment thread docs/MANAGED-IMAGES.md

Comment thread templates/config-repository/fleet.schema.json Outdated

Comment thread templates/config-repository/scripts/validate.py

Comment thread docs/MANAGED-IMAGES.md Outdated

Nicks Hermes added 2 commits August 17, 2026 09:40

fix: stage managed image enforcement safely

1c4dcf3

test: use non-placeholder image IDs in CI

9ba44a6

chatgpt-codex-connector Bot reviewed Aug 17, 2026

View reviewed changes

Comment thread templates/config-repository/scripts/validate.py

Comment thread scripts/install-worker-controller.sh Outdated

fix: keep image staging manager-only

7ca616f

Uh oh!

Conversation

Nickfost commented Aug 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Validation

Remaining gate

Uh oh!

Nickfost commented Aug 16, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Nickfost commented Aug 16, 2026

Uh oh!

chatgpt-codex-connector Bot commented Aug 16, 2026

Uh oh!

Nickfost commented Aug 16, 2026

Uh oh!

chatgpt-codex-connector Bot commented Aug 16, 2026

Uh oh!

Nickfost commented Aug 17, 2026

Uh oh!

chatgpt-codex-connector Bot commented Aug 17, 2026

Uh oh!

Nickfost commented Aug 17, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Nickfost commented Aug 17, 2026

Uh oh!

chatgpt-codex-connector Bot commented Aug 17, 2026

Uh oh!

Nickfost commented Aug 17, 2026

Uh oh!

chatgpt-codex-connector Bot commented Aug 17, 2026

Uh oh!

Nickfost commented Aug 17, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Nickfost commented Aug 17, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

Nickfost commented Aug 17, 2026

Uh oh!

chatgpt-codex-connector Bot commented Aug 17, 2026

💡 Codex Review

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Nickfost commented Aug 16, 2026 •

edited

Loading