feat(ci): dashboard TLS + kill /grind + multi-org dashboard sections - #67
Open
charlielye wants to merge 4 commits into
Open
feat(ci): dashboard TLS + kill /grind + multi-org dashboard sections#67charlielye wants to merge 4 commits into
charlielye wants to merge 4 commits into
Conversation
Remove the /grind Flask route: it built a shell command from URL query params (only cmd was shlex.quoted; commit/cpus/time/etc. interpolated raw into bash -c), an unauthenticated-by-default RCE on the ci host. It was unused, and wired to a stale /aztec-packages checkout. Flaky tests are still ground from the CLI via ./ci.sh grind-test; drop the now-dead grind link in run_test_cmd. Add a docker-compose.yml + Caddyfile so the dashboard runs behind Caddy (TLS via Let's Encrypt) in one stack, replacing the hand-written docker-run systemd unit that lived only on the host. gunicorn binds loopback; Caddy terminates 443. Secrets move to /etc/rkapp.env (the leaked GH_TOKEN must be rotated). Cutover steps are in the compose header.
The dashboard was monorepo-shaped: runs grouped into flat sections (prs, next, releases...) with no repo axis, so multiple orgs/repos collide. Now: - log_ci_run keys each section ZSET as ci-run-<org>/<repo>/<section> (from GITHUB_REPOSITORY), adds a repo field to the blob for metrics, puts a sliding 90-day TTL on the ZSET, and self-registers the dashboard string in a ci-run-sections index set. - rk.py landing page enumerates that index (pruning any section whose ZSET has expired — retired >90d), groups by org/repo, and links each. /section takes a <path:> so slashed dashboard names route. get_section_data is unchanged (ci-run-<name> already). - CI_DASHBOARD classifier (ci.sh multi_job_run): mainline branch -> its own name (main, v5), any tag -> tags, else prs. releases->tags and deflake->local renames complete the section model. Back-compat: callers without GITHUB_REPOSITORY still write ci-run-<section>; old unscoped keys age out via the new TTL and are ignored by the index-driven landing page. The trigger runner's CI_DASHBOARD (still 'prs') is the one coupled follow-up, on the trigger branch where runner.py lives.
deploy.sh now rolls out the compose stack (rkapp on loopback + Caddy TLS): rsync, build the image first, retire the legacy systemd rkapp unit (idempotent, so steady-state redeploys skip it), compose up, liveness-check. Requires /etc/rkapp.env (app secrets) to already exist. Drop the dead /aztec-packages mount from the compose (REPO_PATH/grind are gone). ci.sh log fallback fetches over https, since 80 now redirects and curl won't resend inline basic-auth across an http->https redirect.
… branch The section derivation was duplicated between ci.sh's multi_job_run and log_ci_run's fallback, and the two disagreed (log_ci_run didn't know about mainline-branch or vN sections). The trunk name was also hardcoded and drifted: ci.sh used 'main' while source_refname used 'master' for CI_FULL. Add two shared helpers in source_refname: - ci_default_branch: resolves the repo's trunk (DEFAULT_BRANCH env override -> local origin/HEAD -> git ls-remote --symref -> 'main'). The remote query is what makes it work on the build container's shallow fetch, where origin/HEAD is never set, and it is host-agnostic (GitHub, Forgejo). - ci_dashboard_section: the single source of truth for the section — 'tags' for a semver tag, the ref's own name for the default branch or a configured mainline branch (CI_MAINLINE_BRANCHES, default ^v[0-9]+$), else 'prs'. ci.sh and log_ci_run now both call ci_dashboard_section, and CI_FULL keys off ci_default_branch instead of the stale literal 'master'.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two related dashboard changes, both hardening/generalising the CI dashboard now that it serves multiple orgs and repos.
1. Kill the
/grindweb endpoint + front the dashboard with TLS/grindFlask route.docker-compose.yml+Caddyfileso the dashboard runs behind Caddy (Let's Encrypt TLS) in one stack, replacing the hand-writtendocker runsystemd unit that lived only on the host. gunicorn binds loopback; Caddy terminates 443. Secrets move to/etc/rkapp.env. Cutover steps are in the compose header. (Needs the 443 SG rule — separate iac change.)2. Multi-org dashboard sections
The dashboard was monorepo-shaped: flat sections (
prs,next,releases…) with no repo axis, so multiple orgs/repos collide and one repo's runs evict another's under the 1000-entry cap.log_ci_run: section ZSETs are keyedci-run-<org>/<repo>/<section>(fromGITHUB_REPOSITORY); adds arepofield to the run blob for metricsGROUP BY; puts a sliding 90-day TTL on the ZSET; self-registers the dashboard string in aci-run-sectionsindex set.rk.pylanding page enumerates that index (pruning any section whose ZSET has expired — retired >90d), groups by org/repo, links each./section/<path:section>for the slashes.get_section_dataunchanged.ci.shCI_DASHBOARDclassifier: mainline branch → its own name (main,v5), any tag →tags, elseprs.releases→tags,deflake→local.Back-compat: callers without
GITHUB_REPOSITORYstill writeci-run-<section>; old unscoped keys age out via the new TTL and are ignored by the index-driven landing page.Follow-up (not in this PR)
The trigger service's
runner.pystill setsCI_DASHBOARD="prs", so forge-triggered runs show under<repo>/prsuntil a small ref-classifier lands there (that file is on the trigger branch, not main). The dashboard infra here is ready for it.