Skip to content

fix: complete worker host preflight #71

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Nickfost wants to merge 2 commits into
base: main
Choose a base branch
from
Open

fix: complete worker host preflight #71

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
349821e
fix: complete worker host preflight
Aug 16, 2026
89d7301
fix: bind worker recovery to local Docker
Aug 16, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions docs/DESIRED-STATE.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,8 @@ GitHub App and runner-group creation remain the bootstrap responsibility tracked

## Install a fresh controller

The managed installer supports Debian 12 or newer. Before reading configuration or changing the host, it verifies Docker Engine and Compose v2, Git, curl, jq, the system CA bundle, direct Docker-socket access, and that the Docker filesystem is below the documented 80% warning threshold.

Run the command from a reviewed checkout of ci-fleet on the target Linux Docker machine:

```bash
Expand Down
19 changes: 18 additions & 1 deletion scripts/install-worker-controller.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -135,12 +135,29 @@ cleanup_temporary() {
trap cleanup_temporary EXIT

require_commands() {
local command
local command docker_root disk_used os_id os_release os_version socket
for command in git python3 docker tar install cmp readlink systemctl stat awk grep date flock mktemp; do
command -v "$command" >/dev/null || die "$command is required"
done
docker info >/dev/null 2>&1 || die 'Docker daemon is unavailable'
docker compose version >/dev/null 2>&1 || die 'Docker Compose v2 is unavailable'
[[ "$mode" == rollback || "$mode" == uninstall ]] && return
Comment thread
Nickfost marked this conversation as resolved.

for command in curl jq df; do command -v "$command" >/dev/null || die "$command is required"; done
os_release=$(root_path /etc/os-release)
[[ -r "$os_release" ]] || die 'supported Linux release metadata is unavailable'
os_id=$(awk -F= '$1 == "ID" {gsub(/"/, "", $2); print $2}' "$os_release")
os_version=$(awk -F= '$1 == "VERSION_ID" {gsub(/"/, "", $2); print $2}' "$os_release")
[[ "$os_id" == debian && "$os_version" =~ ^[0-9]+$ ]] || die 'supported Linux is Debian 12 or newer'
((10#$os_version >= 12)) || die 'supported Linux is Debian 12 or newer'
[[ -r $(root_path /etc/ssl/certs/ca-certificates.crt) ]] || die 'CA certificate bundle is unavailable'
socket=$(root_path /var/run/docker.sock)
[[ -S "$socket" && -r "$socket" && -w "$socket" || "$testing" == 1 && -e "$socket" ]] || die 'Docker socket is unavailable or inaccessible'
Comment thread
Nickfost marked this conversation as resolved.
Outdated
docker_root=$(docker info --format '{{.DockerRootDir}}' 2>/dev/null) || die 'Docker root directory is unavailable'
[[ "$docker_root" == /* ]] || die 'Docker root directory is invalid'
disk_used=$(df -P "$docker_root" 2>/dev/null | awk 'NR == 2 {gsub(/%/, "", $5); print $5}')
[[ "$disk_used" =~ ^[0-9]{1,3}$ ]] || die 'Docker disk capacity could not be determined'
((disk_used < 80)) || die 'Docker filesystem must remain below 80% utilization'
}

validate_common_arguments() {
Expand Down
37 changes: 34 additions & 3 deletions scripts/test-install-worker-controller.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,8 @@ export REAL_TAR
REAL_TAR=$(command -v tar)
export REAL_GIT
REAL_GIT=$(command -v git)
export REAL_DF
REAL_DF=$(command -v df)

cat >"$fake_bin/docker" <<'EOF'
#!/usr/bin/env bash
Expand All @@ -24,7 +26,10 @@ state=${FAKE_DOCKER_STATE:?}
status_file=${FAKE_CONTROLLER_STATUS_FILE:-}
paused_state=${FAKE_PAUSED_STATE:-}
case "${1:-}" in
info) exit 0 ;;
info)
[[ "$*" != *DockerRootDir* ]] || printf '%s\n' "${CI_FLEET_DOCKER_ROOT:?}"
exit 0
;;
inspect)
[[ -f "$state" ]] || exit 1
if [[ "$*" == *'.Config.Env'* ]]; then
Expand Down Expand Up @@ -185,6 +190,17 @@ exec "$REAL_GIT" "$@"
EOF
chmod 700 "$fake_bin/git"

cat >"$fake_bin/df" <<'EOF'
#!/usr/bin/env bash
if [[ -n ${FAKE_DISK_USED_PERCENT:-} ]]; then
printf 'Filesystem 1024-blocks Used Available Capacity Mounted on\n'
printf 'fixture 100 90 10 %s%% /fixture\n' "$FAKE_DISK_USED_PERCENT"
exit 0
fi
exec "$REAL_DF" "$@"
EOF
chmod 700 "$fake_bin/df"

export PATH="$fake_bin:$PATH"
export FAKE_DOCKER_STATE=$tmp/docker-controller-running
export FAKE_CONTROLLER_STATUS_FILE=$tmp/docker-controller-status
Expand Down Expand Up @@ -285,7 +301,10 @@ PY
root=$tmp/host
export CI_FLEET_ROOT_PREFIX=$root
export CI_FLEET_DOCKER_ROOT=$root/var/lib/docker
mkdir -p "$root/etc/ci-fleet/secrets" "$CI_FLEET_DOCKER_ROOT"
mkdir -p "$root/etc/ci-fleet/secrets" "$root/etc/ssl/certs" "$root/var/run" "$CI_FLEET_DOCKER_ROOT"
printf 'ID=debian\nVERSION_ID="12"\n' >"$root/etc/os-release"
printf 'fixture CA bundle\n' >"$root/etc/ssl/certs/ca-certificates.crt"
: >"$root/var/run/docker.sock"
pem=$root/etc/ci-fleet/secrets/github-app.pem
printf 'fixture only\n' >"$pem"
chmod 600 "$pem"
Expand All @@ -311,10 +330,19 @@ git -C "$config_repo" reset -q --hard "$ref_one"
installer=$repo_root/scripts/install-worker-controller.sh
base_args=(--config-repo "$config_repo" --controller example-ci-01)

printf 'ID=example\nVERSION_ID="1"\n' >"$root/etc/os-release"
expect_failure 'supported Linux is Debian 12 or newer' "$installer" --check "${base_args[@]}" --ref "$ref_one"
printf 'ID=debian\nVERSION_ID="12"\n' >"$root/etc/os-release"
export FAKE_DISK_USED_PERCENT=80
expect_failure 'Docker filesystem must remain below 80% utilization' "$installer" --check "${base_args[@]}" --ref "$ref_one"
unset FAKE_DISK_USED_PERCENT

staged_checkpoint="$root/var/lib/ci-fleet/checkpoints/.checkpoint.staging.interrupted"
mkdir -p "$staged_checkpoint"
: >"$staged_checkpoint/.complete"
mv "$root/etc/os-release" "$root/etc/os-release.missing"
expect_failure 'no controller checkpoint is available' "$installer" --rollback
mv "$root/etc/os-release.missing" "$root/etc/os-release"
rm -rf "$staged_checkpoint"
expect_failure 'secret-bearing files are forbidden' "$installer" --check "${base_args[@]}" --ref "$forbidden_ref"
expect_failure 'possible committed secret detected' "$installer" --check "${base_args[@]}" --ref "$secret_ref"
Expand Down Expand Up @@ -636,7 +664,10 @@ unset FAKE_RUNNER_STATE_ONCE FAKE_ALL_RUNNER_STATE
adopt_root=$tmp/adopt-host
export CI_FLEET_ROOT_PREFIX=$adopt_root
export FAKE_DOCKER_STATE=$tmp/adopt-controller-running
mkdir -p "$adopt_root/etc/ci-fleet/secrets" "$adopt_root/opt/ci-fleet/deploy" "$adopt_root/opt/ci-fleet/scripts"
mkdir -p "$adopt_root/etc/ci-fleet/secrets" "$adopt_root/etc/ssl/certs" "$adopt_root/var/run" "$adopt_root/opt/ci-fleet/deploy" "$adopt_root/opt/ci-fleet/scripts"
printf 'ID=debian\nVERSION_ID="12"\n' >"$adopt_root/etc/os-release"
printf 'fixture CA bundle\n' >"$adopt_root/etc/ssl/certs/ca-certificates.crt"
: >"$adopt_root/var/run/docker.sock"
adopt_pem=$adopt_root/etc/ci-fleet/secrets/github-app.pem
printf 'fixture only\n' >"$adopt_pem"
chmod 600 "$adopt_pem"
Expand Down