diff --git a/.github/workflows/ci-cloud-image.yml b/.github/workflows/ci-cloud-image.yml index cd5986e..3e8bf41 100644 --- a/.github/workflows/ci-cloud-image.yml +++ b/.github/workflows/ci-cloud-image.yml @@ -171,7 +171,8 @@ jobs: # mkosi-sandbox EPERM (#189), so the fix has to be proven here, not on a # floating ubuntu-latest. runs-on: ubuntu-24.04 - # Two mkosi builds (production image + QEMU test image), three QEMU boots, and + # Two mkosi builds (production image + QEMU test image), the QEMU boots (five + # on a push/dispatch, just `update` on a PR — see MALMO_CLOUD_BOOTS below), and # the compress-and-attach publish run ~10 min in practice; the 120 cap is a # generous ceiling so a one-off slow runner is never killed mid-build. timeout-minutes: 120 diff --git a/CLAUDE.md b/CLAUDE.md index f522ba5..5ebfffe 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -12,7 +12,7 @@ Two phrases that constrain a lot of design: A running malmo is five processes/artifacts. Three are Go, one is JavaScript, one is a container we don't write. - **`malmo-brain`** (`cmd/brain/`, `internal/`) — the control-plane daemon. One Go binary: owns SQLite state, the REST+SSE API, the app lifecycle, and the Caddy config. Drives Docker via the `docker compose` CLI. -- **`host-agent`** (`cmd/host-agent/`) — the privileged side. Today it's a **fake**: real `BRAIN_HOST_PROTOCOL.md` wire format over a real UNIX socket, but the host ops themselves (Avahi, LUKS, PAM, apt) are stubbed in memory. The real one is `cmd/host-agent-real/` (PAM verify is real; the rest is still being built). +- **`host-agent`** — the privileged side. Two binaries. `cmd/host-agent/` is the **fake** used by the inner dev loop: it speaks the real `BRAIN_HOST_PROTOCOL.md` wire format over a real UNIX socket, but the host ops are stubbed in memory. `cmd/host-agent-real/` is the real one, and most of it works now: PAM verify, user management, `/proc` sampling, disk and RAM, journal streaming, service health, reboot-required, time zone, Avahi discovery, the first-boot brain launch, and the control-plane update path. Not wired yet: LUKS/TPM, apt, and NetworkManager config. It also builds a slim `hosted` variant (`go build -tags hosted`) for the cloud image. See `docs/architecture.md` # Components for the current split. - **`web-ui`** (`web-ui/`) — Vue 3 + Vite + TanStack Query dashboard. Talks only to the brain. - **Caddy** (`dev/`) — reverse proxy. Terminates `*.local` and routes to app containers + the brain, configured live by the brain via Caddy's admin API. Subdomain routing per app, never path-based (browser same-origin policy is the reason). - **SQLite** — the brain's only persistent store (`internal/store/`). @@ -24,8 +24,8 @@ Wire: `browser → web-ui → brain`, and the brain fans out to `docker compose` ## Where the important files are - **`cmd/brain/main.go`** — ~100 lines; names every package and how they wire. Best single starting point for the code. -- **`internal/`** — the brain's packages (`api`, `lifecycle`, `store`, `catalog`, `manifest`, `admission`, `caddy`, `hostclient`, `protocol`, `auth`, `audit`, `events`, plus host-integration and health packages). What each owns and the import rules are in `docs/architecture.md` # Inside the brain. -- **`cmd/`** — entry points: `brain`, `host-agent` (fake), `host-agent-real`, plus small tools (`malmo`, `malmo-storage-verify`, `openapi-gen`). +- **`internal/`** — the brain's packages (`api`, `lifecycle`, `store`, `catalog`, `manifest`, `admission`, `caddy`, `profile`, `hostclient`, `protocol`, `auth`, `assertion`, `audit`, `events`, `version`, plus the health/observability set — `health`, `notify`, `applog`, `systemlive`, `storageverify` — and `internal/hostagent/…`, the host-side implementation packages). What each owns and the import rules are in `docs/architecture.md` # Inside the brain. +- **`cmd/`** — entry points: `brain`, `host-agent` (fake), `host-agent-real`, plus small tools (`malmo`, `malmo-storage-verify`, `malmo-network-verify`, `openapi-gen`). - **`web-ui/`** — the dashboard. Internal code architecture in `docs/dev/web-ui.md`. - **Catalog apps** — not in this repo, and they don't get added here. The artifacts (`manifest.yml`, `compose.yml`, icons, screenshots) live in `malmoos/store` under `apps//`, the control plane publishes them as one snapshot, and a box pulls that snapshot from a malmo endpoint at runtime — it keeps no copy on disk (cloud #62, `DECISIONS.md` 2026-07-02 and 2026-08-17). **Adding an app starts with a Catalog app issue in `malmoos/store`**, not here; authoring works from that issue per `docs/dev/authoring-apps-with-an-agent.md`, which keeps the schema (`internal/manifest`), the admission policy, the `malmo manifest` CLI, and the gap ledger on this side. Catalog test fixtures in this repo are **synthetic** — fake apps in the published wire shape, never a copy of a snapshot the endpoint serves (`internal/catalog/testdata/snapshot.json`). - **`Makefile` + `dev/`** — dev orchestration (`make help`). @@ -61,7 +61,7 @@ The inner/outer boundary is also the **cross-platform / Linux-only** boundary. T - `make test-nopam` — full suite minus the PAM package, for when you don't have `libpam0g-dev` (i.e. off Linux). - `make clean` — stop dev Caddy, remove malmo containers/networks, wipe `.dev/state`. -**Building the hosted cloud image is a CI job — don't build it locally.** `make build-cloud-image` / `make test-cloud-qemu` need root + `/dev/kvm` + mkosi and take ~10+ min; a local mkosi build is fragile and easy to get wrong (a broken build is what once produced a phantom "`:443` doesn't bind" hunt). Instead trigger the **`CI / Cloud image`** GitHub Action: `gh workflow run "CI / Cloud image" --ref -f publish=false` builds the image and runs the QEMU boot-proof (`unseeded seeded bios access update` boots) **without** publishing anything. Only `publish=true` (the default) uploads the image to the provider — a deliberate act, not a test. See [`docs/dev/hosted-boot-proof.md`](docs/dev/hosted-boot-proof.md) for reading the result and debugging a red boot. +**Building the hosted cloud image is a CI job — don't build it locally.** `make build-cloud-image` / `make test-cloud-qemu` need root + `/dev/kvm` + mkosi and take ~10+ min; a local mkosi build is fragile and easy to get wrong (a broken build is what once produced a phantom "`:443` doesn't bind" hunt). Instead trigger the **`CI / Cloud image`** GitHub Action: `gh workflow run "CI / Cloud image" --ref -f publish=false` builds the image and runs the QEMU boot-proof (`unseeded seeded bios access update` boots) **without** publishing anything. Only `publish=true` (the default) publishes anything. Since #352 that means two things: it **attaches the compressed image and its checksum to the tagged GitHub Release**, and it **pushes the brain and UI images to ghcr**. It uploads to no hosting provider and holds no provider login. Publishing is a deliberate act, not a test. See [`docs/dev/hosted-boot-proof.md`](docs/dev/hosted-boot-proof.md) for reading the result and debugging a red boot. **Prerequisites for the inner loop:** Docker + `docker compose`, Node 20+, Go 1.23+, host port `:80` free (dev Caddy binds it so `.local` works portless), and `avahi-daemon` running on Linux (so `.local` names resolve under `make dev`). The full Go test suite additionally needs `libpam0g-dev` on Linux; see `docs/dev/running-locally.md`. @@ -83,7 +83,7 @@ Every change ships with documentation — a code change is not complete until it - **Root `README.md`** is the front door (pitch + quickstart); keep its quickstart accurate when the dev workflow changes. - **No line wrapping in markdown.** Use continuous lines of text, not ~70-character breaks. Markdown viewers reflow; hard-wrapped lines make diffs harder to read. -`DECISIONS.md` (evolution-of-thinking log — read before relitigating) and `NEXT.md` (prioritized open design topics; the only place open items live — never add them to individual docs) are the two cross-cutting docs to know by name. +Two cross-cutting docs to know by name. `docs/specs/DECISIONS.md` is the log of how our thinking changed — read it before you reopen a settled question. `docs/specs/NEXT.md` holds the open design topics, in priority order; it is the only place open items live, so never add them to individual docs. Both sit in `docs/specs/`, and there is no copy of either at the repo root. Specs link to them by bare filename (`NEXT.md`) because every spec is their sibling. ## Go code discipline @@ -92,7 +92,7 @@ Small set of rules. Codified now so we don't have to back them out later. - **Consumer-side interfaces.** Interfaces live in the package that *uses* them, not the package that implements them. `lifecycle.DockerDriver` lives in `internal/lifecycle/`, not in a hypothetical `internal/docker/`. Provider packages export concrete types only. Exception: a single interface shared by three or more consumers can move to the provider, but default to consumer-side until that's true. - **Layer boundaries.** `internal/lifecycle` is the transaction owner; only `cmd/brain` and `internal/api` may import it. `internal/store` is the persistence boundary; only `internal/lifecycle`, `internal/api`, `internal/auth`, `internal/audit`, and `cmd/brain` may import it. Anything else reaching in is breaking the model — push the call through the right seam instead. - **`log/slog` is the only logger.** No `"log"` imports, no `fmt.Println` for diagnostics. Structured fields, not interpolated strings: `slog.Info("app installed", "instance_id", id)`, not `slog.Info(fmt.Sprintf("installed %s", id))`. The default handler is set in `cmd/brain/main.go`; use `slog.Default()` (the package-level functions) — don't thread `*slog.Logger` through constructors. -- **Standard structured fields.** Use these key names so journalctl/jq filters stay stable: `instance_id`, `manifest_id`, `slug`, `service`, `image`, `host`, `upstream`, `step`, `err`, `output`, `user_id`, `username`, `role`, `action`, `actor_user_id`, `target_kind`, `target_id`, `retry_after`, `iface`, `interfaces`, `src`, `dir`, `profile`, `box_id`, `zone`, `exposure`, `trusted_proxies`, `brain`, `ui`, `minimum_host_agent`, `keys`, `state_dir`, `job_id`, `window`, `url`, `from`. `host` is a machine or upstream hostname only — a single network interface name is `iface`, a list of them is `interfaces` (never overload `host` for either). `src` is a source filesystem path (bind-source, folder-source); `dir` is a relative bind dir path. `profile` is the resolved environment profile (`appliance`|`hosted`, `ENVIRONMENT.md`). `box_id` is the hosted box's provisioned identity (`ENVIRONMENT.md` # Provisioning). `zone` is an IANA time-zone name (`TIME.md`, host-agent set-timezone). `exposure` is an app's per-instance access mode (`restricted`|`public`, `ENVIRONMENT.md` #306). `trusted_proxies` is the configured set of proxies whose `X-Forwarded-For` the brain reads when deriving a client IP (`BRAIN_UI_PROTOCOL.md` # Rate limiting & abuse). `brain` and `ui` are the two control-plane versions a release names, and `minimum_host_agent` the host-agent version it requires (`RELEASE_MANIFEST.md`); use them for versions, not for image refs — an image ref is `image`. `keys` is how many signing keys a build accepts, and `state_dir` a state directory path (`src` stays for a source path being read or bound). `job_id` is a host-agent job id (`internal/hostagent/jobs.go`), and `window` the configured update window (`UPDATES.md` # 8.4). `url` is an HTTP endpoint the box reads, and `from` says which of several configured sources a setting came from, one of `answer` (the control plane's update-target answer), `seed`, `env`, `default` (`UPDATES.md` # 8.4); where there are only two sources, the boolean `from_` form is used instead (`from_ledger`). Adding a new recurring field? Add it here. +- **Standard structured fields.** Use these key names so journalctl/jq filters stay stable: `instance_id`, `manifest_id`, `slug`, `service`, `image`, `host`, `upstream`, `step`, `err`, `output`, `user_id`, `username`, `role`, `action`, `actor_user_id`, `target_kind`, `target_id`, `retry_after`, `retry_after_s`, `name`, `uid`, `iface`, `interfaces`, `src`, `dir`, `profile`, `box_id`, `zone`, `exposure`, `trusted_proxies`, `brain`, `ui`, `minimum_host_agent`, `keys`, `state_dir`, `job_id`, `window`, `url`, `from`. `host` is a machine or upstream hostname only — a single network interface name is `iface`, a list of them is `interfaces` (never overload `host` for either). `src` is a source filesystem path (bind-source, folder-source); `dir` is a relative bind dir path. `profile` is the resolved environment profile (`appliance`|`hosted`, `ENVIRONMENT.md`). `box_id` is the hosted box's provisioned identity (`ENVIRONMENT.md` # Provisioning). `zone` is an IANA time-zone name (`TIME.md`, host-agent set-timezone). `exposure` is an app's per-instance access mode (`restricted`|`public`, `ENVIRONMENT.md` #306). `trusted_proxies` is the configured set of proxies whose `X-Forwarded-For` the brain reads when deriving a client IP (`BRAIN_UI_PROTOCOL.md` # Rate limiting & abuse). `brain` and `ui` are the two control-plane versions a release names, and `minimum_host_agent` the host-agent version it requires (`RELEASE_MANIFEST.md`); use them for versions, not for image refs — an image ref is `image`. `keys` is how many signing keys a build accepts, and `state_dir` a state directory path (`src` stays for a source path being read or bound). `job_id` is a host-agent job id (`internal/hostagent/jobs.go`), and `window` the configured update window (`UPDATES.md` # 8.4). `url` is an HTTP endpoint the box reads, and `from` says which of several configured sources a setting came from, one of `answer` (the control plane's update-target answer), `seed`, `env`, `default` (`UPDATES.md` # 8.4); where there are only two sources, the boolean `from_` form is used instead (`from_ledger`). `name` is an app instance's display name (it rides alongside `instance_id`, never instead of it), and `uid` a numeric Unix user id — the allocated app-service identity or a resolved home owner. `retry_after` and `retry_after_s` come from the two throttles that `AUTH.md` # Rate limiting keeps apart on purpose. Both are correct. Do not merge them. `retry_after` is the login backoff's wait, written as a duration string (`internal/api/auth.go`); that path sends no `Retry-After` header, by design. `retry_after_s` is the general request limiter's wait, written as a whole number of seconds; it matches that limiter's `retry_after_s` JSON field and the `Retry-After` header it sets (`internal/api/ratelimit.go`, `BRAIN_UI_PROTOCOL.md` # 429 contract). To find every throttled request you must search for both keys. That is the price of the split, not a bug. Adding a new recurring field? Add it here. - **Typed errors at boundaries, not everywhere.** Define a sentinel/typed error only when a *consumer* needs to discriminate (HTTP status, retry decision, UI text). `store.ErrNotFound` exists because the API maps it to 404. Don't pre-declare error types speculatively. - **No premature abstraction.** Don't introduce an interface, factory, or DI container until at least two concrete consumers exist. It bites hardest in Go where every extra interface is import-graph weight. - **`internal/` for everything except `cmd/`.** No `pkg/`. Anything inside `internal/` is private to this module by Go's own rules — no public API surface to maintain. @@ -108,7 +108,7 @@ Small set of rules. Codified now so we don't have to back them out later. - **ext4 + LUKS, not ZFS.** ZFS forecloses mergerfs/SnapRAID upgrades and adds CDDL/kernel licensing pain. - **Mergerfs from day 1** when a data drive is present (pool of one with one drive; `epmfs` placement). Enables zero-downtime drive addition. SnapRAID parity stays deferred. - **User content at `/home//`** with macOS-style capitalized use-case folders (`Photos/`, `Music/`, `Movies/`, `Documents/`, `Notes/`, `Downloads/`). Data drive mounts at `/srv/malmo/` with bind mounts to `/home/` and `/var/lib/malmo/`. -- **Files are first-class, apps are windows.** User content lives in use-case folders; app state in `/var/lib/malmo/instances//`. Uninstalling an app never deletes user content. Manifests bind-mount use-case folders by declaration. +- **Files are first-class, apps are windows.** User content lives in use-case folders; app state in `/var/lib/malmo/state/instances//`. Uninstalling an app never deletes user content. Manifests bind-mount use-case folders by declaration. - **SMB shares via Samba** for cross-device access (Windows, macOS, iOS, Android, Linux). mDNS-advertised. TimeMachine-compatible. - **Avahi as the LAN publisher; per-app `.local` records owned by the reconciler.** No mDNS wildcards exist; each app slug is a real announced name, published by host-agent via Avahi DBus `EntryGroup.AddAddress` alongside the Caddy site block. LAN interfaces only (not mesh, not Docker bridges). `.local` is HTTP-only by definition (no public DNS → no Let's Encrypt) and Android browsers don't resolve it — secure URLs are the compatibility path. See `DISCOVERY.md`. - **One malmo password per user, PAM is the source of truth.** Dashboard, SSH, and SMB all authenticate against the same `/etc/shadow` entry. Brain has no password hash; it calls host-agent's `verify_password` on every login. Per-protocol opt-in (SSH and SMB off-by-account-by-default) is done via service allowlists, not separate credentials. @@ -127,6 +127,6 @@ Small set of rules. Codified now so we don't have to back them out later. - **Write in plain English — CEFR B1.** This applies to everything you produce: chat replies, docs, progress entries, PR bodies, issue text, commit messages, code comments, and UI copy. Short sentences, common words, one idea per sentence. Prefer "use" over "utilise", "let" over "facilitate", "so" over "consequently". Cut clause-stacking and em-dash pile-ups; split into two sentences instead. Technical terms the project owns (`manifest`, `reconciler`, `bind mount`, `PGDATA`) stay — B1 is about the sentence around the term, not about dumbing down the domain. Precision wins if the two ever conflict: say the exact thing in simple words rather than a vague thing in simple words. - Read the relevant `docs/specs/` doc(s) end-to-end before proposing changes — they cross-reference each other heavily and decisions in one constrain the others. Use `docs/README.md` to find the right one. - Push back on tradeoffs; defer to product calls once made (per user preference). -- Open questions are tracked at the bottom of each doc and in `NEXT.md` — that's where genuinely unresolved items live. Don't invent answers; surface them. +- Open questions are tracked at the bottom of each doc and in `docs/specs/NEXT.md` — that's where genuinely unresolved items live. Don't invent answers; surface them. - Don't add "future-proofing" abstractions to the spec. The docs are already explicit about what's deferred (e.g., fscrypt, ARM, snapshots, paid-app mechanics). - Keep the "no NAS vocabulary in the UI" rule (`STORAGE.md`) in mind for any user-facing language. diff --git a/NEXT.md b/NEXT.md deleted file mode 100644 index 87fd285..0000000 --- a/NEXT.md +++ /dev/null @@ -1,37 +0,0 @@ -# Next - -Prioritized open design topics. The only place open items live — never add them to individual docs. - -## `malmo resolve`: daemon-free registry sizer for `disk_bytes` - -PR #120 (fix/117) fixed the containerd-store bug by streaming `docker save` and decompressing layer blobs locally. It works and is store-agnostic, but it is slow — a multi-GB image like open-webui takes ~a minute to stream and decompress just to count bytes. - -The clean path: fetch the image manifest directly from the registry (no local pull, no `docker save`), walk the layer descriptors, fetch each compressed blob, decompress on the fly, and sum the bytes. Same decompressed-tar number, no Docker daemon required, and it is the natural home for catalog CI where no daemon is available anyway. The registry client work was already scoped in [catalog-image-footprint.md](docs/progress/catalog-image-footprint.md) — this is the next concrete step on that path. - -## Admin-port isolation: Caddy connects outward to per-app networks ([#187](https://github.com/malmoos/malmo/issues/187)) - -App `main_service` containers join `malmo-ingress` (`internal/lifecycle/lifecycle.go`), the network that also carries Caddy's unauthenticated admin API (`0.0.0.0:2019`). `CONTROL_PLANE.md` # Locked: Caddy is malmo substrate makes this a hard invariant violation: a compromised app could rewrite the entire Caddy route table via `malmo-caddy:2019`. Harmless in dev, but M1b (#165) makes it production-real. - -There is no small fix — Caddy's admin must bind `0.0.0.0:2019` (the brain is a separate container), so it is exposed on every network Caddy joins. The spec-prescribed model is the fix: apps stay **off** `malmo-ingress`, and the brain connects `malmo-caddy` **outward** into each `malmo-app-` network (with reconnect-on-Caddy-restart in `Reconcile` and disconnect-before-teardown). Must land before any production exposure; v1 is closed-by-default / pre-production, so nothing is live in the interim. Tracked in #187. - -## Public acme-dns API face for hosted boxes (cross-repo, gates #207 acceptance) — DEPLOYED - -C3b (#207, [hosted-wildcard-cert.md](docs/progress/hosted-wildcard-cert.md)) configures the hosted box's Caddy to push its `_acme-challenge` TXT to an acme-dns endpoint — a box-side constant `MALMO_ACMEDNS_ENDPOINT`, default `https://auth.malmo.network`. **Deployed and confirmed (2026-06-23, `malmoos/cloud` #14):** the cloud control-plane VM now fronts acme-dns with Caddy, exposing only `/update` + `/health` over real Let's Encrypt TLS for `auth.malmo.network` (404 on `/register`, which stays loopback-only), with the authoritative `:53` face delegated and answering publicly. The default `https://auth.malmo.network` is therefore a **confirmed live endpoint**, not a chosen value — the box-side default needs no change. The one remaining piece is the joint real-issuance run: a live box actually obtaining/renewing its `*..malmo.network` wildcard against this endpoint, verified at cloud #6 / CL6 (not an OS-side task — the box-side wiring is complete). - -## Authenticate the box to the control plane (the update-target ask, and everything after it) - -A hosted box now says which box it is when it asks for its update target: `GET ?box_id=` (#408, `UPDATES.md` # 8.1). The endpoint is public and unauthenticated, so the identity is a claim, not a proof. Anyone who learns a box-id can read what that box is told to run, and can ask as that box. - -That is accepted while the ask is only a **read** of version information, and while the operator path that sets a target sits behind the control plane's own auth. It stops being enough the moment the box **writes**: the report-back in `UPDATES.md` # 8.4 step 5 ("this box is now on v0.7.0, or it failed and rolled back") and the fleet-side auto-halt in # 8.5 both need the control plane to trust who is talking. A fleet view built on an unauthenticated report is a fleet view anyone can lie to. - -What is missing is a general box↔cloud credential. The `enrollment` block in `seed.json` is scoped to acme-dns and cannot be reused for this. The seed is the delivery channel — it is the only per-box channel a real box has, and it is write-once (`ENVIRONMENT.md` # Provisioning & first-boot) — so a long-lived per-box secret with an on-box rotation path is the shape to design. Nothing that mutates state should be built on the bare `box_id` in the meantime. - -## Encrypt hosted enrollment credentials at rest (box-side) - -The per-box acme-dns credentials the brain ingests from the seed are persisted plaintext in `box_meta` (`store.BoxMetaEnrollment`), matching the cloud producer's MVP posture. The threat is loss of the brain's SQLite: a leaked subdomain/username/password lets an attacker renew certs for that one box, not escalate beyond it. Encrypt at rest when the box's DB lands on shared/backed-up infra. The cloud side tracks the symmetric item for its own `boxes` table (`malmoos/cloud` NEXT.md). - -## Per-app disk quota for hosted tenants - -`ENVIRONMENT.md` # Per-instance resource limits names a per-app **disk quota** as the third dimension the hosted control plane needs to bound a paying tenant. The memory + CPU cgroup limits landed with #211, but disk quota was deferred: the locked storage stack (`ext4` + Docker's `overlay2`, `STORAGE.md`) cannot enforce a per-container write-layer quota portably. Docker's `--storage-opt size=` works only on `xfs`-with-`pquota` or the `devicemapper`/`btrfs` drivers — none of which the appliance or the cloud image use. - -The realistic path is **XFS project quotas** on the data tree, driven through the host-agent (a privileged op — set/clear a project ID + hard limit on an app's `/var/lib/malmo/instances//` subtree and its bound use-case folders). It needs: a `BRAIN_HOST_PROTOCOL.md` verb, a `host-agent-real` implementation, and the same store-backed-policy + reconcile seam the cgroup limits already use (`internal/store` `instance_resource_limits` would gain a `disk_bytes` column). Hosted-only in practice; out of scope until the cloud image's storage layout is fixed. Deferred from #211; tracked in #221. diff --git a/README.md b/README.md index 86e9fb3..a5306fd 100644 --- a/README.md +++ b/README.md @@ -56,7 +56,7 @@ What runs today (mostly in the native inner loop, see [Quickstart](#quickstart-l - **Real host-agent (`host-agent-real`).** PAM verify, user create / delete / role / password, real `/proc` sampling, disk and RAM reporting, journal streaming, per-LAN-interface Avahi discovery, and first-boot brain launch (Docker socket proxy + brain container). LUKS/TPM enrollment and the boot-chain units exist and are exercised in the QEMU test lane. - **Hosted cloud profile (coming online).** A slim, build-tagged cloud `host-agent`; a lean `mkosi` cloud image with self-bootstrapping first boot; real Let's Encrypt wildcard certs over ACME DNS-01 for `*..malmo.network` (the `auth.malmo.network` acme-dns face is live); an app-container egress block for the cloud metadata endpoint; and a portal-to-box SSO handshake so the owner reaches the box through their existing `malmo.network` login. The image builds, boots, and provisions on a real cloud provider; a CI lane plus a cloud QEMU lane drive the seed → first-run → served-dashboard arc, with full end-to-end acceptance still being hardened. -What is **not** built yet (so this isn't read as a finished-product claim): the appliance storage subsystem (`/srv/malmo`, mergerfs, the LUKS-unlock-at-boot flow), the production install ISO and the update streams, WiFi/NetworkManager configuration in the agent, and the signed remote app-store fetch. The authoritative as-built map is [`docs/architecture.md`](docs/architecture.md) (# What is not built yet); per-change history is in [`docs/progress/`](docs/progress/). +What is **not** built yet, so nobody reads this as a finished product: the appliance storage subsystem (`/srv/malmo`, mergerfs, and the LUKS unlock at boot beyond the QEMU lane), the production install medium, stream A of updates (the apt and `unattended-upgrades` half), and WiFi/NetworkManager setup in the agent. Stream B is the box updating its own brain and UI. On `hosted` that is built and proven on a booted box. On `appliance` the box can read a signed release manifest, but there is no signing key and no release host yet, so it does nothing on purpose. The authoritative as-built map is [`docs/architecture.md`](docs/architecture.md) (# What is not built yet); per-change history is in [`docs/progress/`](docs/progress/). ## Architecture @@ -78,11 +78,12 @@ browser → web-ui → brain → docker compose (Docker daemon) | Path | What lives here | |---|---| -| `cmd/` | Go entrypoints: `brain`, `host-agent` (fake), `host-agent-real`, plus small tools (`malmo`, `malmo-storage-verify`, `openapi-gen`) | -| `internal/` | brain packages: `api`, `lifecycle`, `store`, `catalog`, `manifest`, `admission`, `caddy`, `hostclient`, `protocol`, `auth`, `audit`, `events`, `profile`, `assertion`, plus host-integration and health packages | +| `cmd/` | Go entrypoints: `brain`, `host-agent` (fake), `host-agent-real`, plus small tools (`malmo`, `malmo-storage-verify`, `malmo-network-verify`, `openapi-gen`) | +| `internal/` | brain packages: `api`, `lifecycle`, `store`, `catalog`, `manifest`, `admission`, `caddy`, `hostclient`, `protocol`, `auth`, `audit`, `events`, `profile`, `assertion`, `version`, the health/observability set (`health`, `notify`, `applog`, `systemlive`, `storageverify`), and `internal/hostagent/…` for the host side | +| `api/` | the generated OpenAPI document (`make openapi`); `make check` fails if it is stale | | `web-ui/` | Vue 3 + Vite dashboard | -| `catalog/` | hand-written sample app manifests (`manifest.yml` + `compose.yml`) | -| `dev/` | local dev orchestration (Caddy container, config, test lanes) | +| `dist/` | systemd units and drop-ins shipped onto a real box | +| `dev/` | local dev orchestration (Caddy container, config, image trees, test lanes) | | `docs/` | all documentation (specs, progress, architecture, dev guides) | | `Makefile` | dev workflow, run `make help` | @@ -97,7 +98,7 @@ make dev # the whole inner-loop stack in one terminal: # Caddy (container) + fake host-agent + brain + Vite ``` -Then open and install **Whoami** from the catalog. `make dev` also publishes each app's `.local` name over real Avahi, so installed apps are reachable by their portless `.local` URL from this box and other LAN devices (Android browsers don't resolve `.local`). Ctrl-C stops everything. +Then open and install **Whoami** from the catalog. (The catalog is not in this repo — the brain syncs it from the control plane at run time. To work against a specific store app instead, use `make dev-app APP=` with a `malmoos/store` checkout.) `make dev` also publishes each app's `.local` name over real Avahi, so installed apps are reachable by their portless `.local` URL from this box and other LAN devices (Android browsers don't resolve `.local`). Ctrl-C stops everything. Prefer separate terminals? Run the pieces individually: diff --git a/VERSION b/VERSION index a918a2a..78bc1ab 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -0.6.0 +0.10.0 diff --git a/docs/README.md b/docs/README.md index 5061d88..b0705b8 100644 --- a/docs/README.md +++ b/docs/README.md @@ -20,14 +20,15 @@ Actionable parallel work lives in [GitHub Issues](https://github.com/malmoos/mal ## Specs `specs/` holds the design docs. [`specs/SPEC.md`](specs/SPEC.md) is the entry -point; the full annotated list (what each doc owns and its headline decisions) -is the **Documents** section of [`../CLAUDE.md`](../CLAUDE.md). Cross-references -inside the specs are bare filenames, relative to `specs/`. +point, and [`../CLAUDE.md`](../CLAUDE.md) holds the big decisions the specs +build on. The list below groups every spec in `specs/`. If a doc is in that +folder and not in this list, that is a bug: fix it in the same change. Inside +the specs, cross-references are bare filenames, relative to `specs/`. Orientation: - **Start here:** `SPEC.md`, `CONTROL_PLANE.md`, `ENVIRONMENT.md` (the two environment profiles — `appliance` vs malmo-operated `hosted` — and every hosted-specific delta). -- **Apps:** `APP_LIFECYCLE.md`, `APP_MANIFEST.md`, `APP_STORE.md`, `APP_ISOLATION.md`, `SERVICE_PROVISIONING.md`. +- **Apps:** `APP_LIFECYCLE.md`, `APP_MANIFEST.md`, `APP_STORE.md`, `APP_ISOLATION.md`, `SERVICE_PROVISIONING.md`, `CAPABILITIES.md` (the machine-readable ledger of shipped platform capabilities, so catalog curation stops depending on someone remembering — the manifest itself is [`dev/capabilities.yml`](dev/capabilities.yml)). - **Protocols:** `BRAIN_UI_PROTOCOL.md`, `BRAIN_HOST_PROTOCOL.md` (Pattern C stream 1 — `journal_follow` per-app log tail — is now implemented; `journal_query` and `journal_export_range` remain deferred). - **Frontend:** `WEB_UI.md` (stack/deploy), `DASHBOARD.md` (logged-in IA + the owner-scoped apps model + install flows, incl. Door-2 custom-container), `SETTINGS.md` (Settings IA: My-account / Box-settings split, panel inventory, role gating), `FILES.md` (in-dashboard file manager). - **System:** `STORAGE.md`, `BOOT.md`, `DISCOVERY.md`, `MALMO_NETWORK.md`, `TIME.md`, `USERS_AND_GROUPS.md`, `AUTH.md`. diff --git a/docs/architecture.md b/docs/architecture.md index b35e0b5..ac64ef7 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -16,7 +16,7 @@ one is JavaScript, one is a container we don't write. | **`host-agent` (fake)** | `cmd/host-agent/` | Privileged side used in the inner dev loop. Speaks the real `BRAIN_HOST_PROTOCOL.md` wire format over a UNIX socket; the host operations themselves (Avahi, LUKS, PAM, apt) are stubbed in memory. | **Fake** (real wire, canned ops) | | **`host-agent-real`** | `cmd/host-agent-real/`, `internal/hostagent/` | The real privileged binary. Seam-injected reporters: PAM password verify (`pamverifier`), `/proc` system sampling (`procsource`), disk usage, RAM pressure, journal streaming, service health, reboot-required flag, user manager, system time-zone setter (`timezone`, `timedatectl set-timezone` — the first-run wizard's Step 3, wired in both build profiles). Discovery is real: per-LAN-interface Avahi announcements (`avahipublisher`) driven by the NetworkManager LAN set (`netstate`), with an avahi-daemon.conf allowlist sync and IP-change replay. Seeds the brain's Docker transport then launches the brain container on startup (`brainlaunch`: `EnsureTransport` creates `malmo-ingress` + runs the `docker-socket-proxy`; `Launch` docker-loads the bundled image if absent, lockstep `malmo.protocol.major` OCI-label check, `docker run --restart unless-stopped` on the ingress net with `DOCKER_HOST` at the proxy). Host ops not yet wired: LUKS/TPM, apt, NM configuration (WiFi setup, `/v1/network/*`). A build-tagged slim **`hosted`** profile (`go build -tags hosted`, #204/C1c) compiles the discovery/NetworkManager stack out for the cloud image — `avahipublisher`/`netstate` unwired, no-op publisher, nil `Net` — keeping the same PAM/user-mgmt/health-system/brain-launch seams (`cmd/host-agent-real/wiring_appliance.go` vs `wiring_hosted.go`). | Partial — see "What is not built yet" | | **Caddy** | `dev/caddy.json`, `dev/docker-compose.yml` | Reverse proxy. Terminates `*.local` (appliance) or `*..malmo.network` over real Let's Encrypt HTTPS (hosted, via a custom acme-dns build) and routes to app containers + the brain. Configured live by the brain via Caddy's admin API. | Real (container) | -| **`web-ui`** | `web-ui/` | Vue 3 + Vite + TanStack Query dashboard. Talks only to the brain. Tailwind 4 landed; shadcn-vue scaffolding present, components not yet copied in. Internal code architecture: [`dev/web-ui.md`](dev/web-ui.md). | Real | +| **`web-ui`** | `web-ui/` | Vue 3 + Vite + TanStack Query dashboard. Talks only to the brain. Tailwind 4 with the Oatmeal `@theme` tokens; `reka-ui` + `cn()` are present as shadcn-vue scaffolding, but the owned components in `components/ui/` (`Button`, `Heading`) are hand-written from the Oatmeal patterns, not pulled through the shadcn CLI (#261). Internal code architecture: [`dev/web-ui.md`](dev/web-ui.md). | Real | | **SQLite** | `$STATE_DIR/malmo.db` | The brain's only persistent store. Schema + queries in `internal/store/`. | Real | Plus the **Docker daemon** on the host, which the brain drives with the @@ -65,16 +65,25 @@ Plus the **Docker daemon** on the host, which the brain drives with the container's health check passes, then flips to the real upstream. - **brain → host-agent:** HTTP/JSON over `MALMO_AGENT_SOCK`. Two patterns, sync request/response and SSE-streamed jobs (`internal/protocol/host.go` - defines the types; `internal/hostclient/` is the brain-side client). Today - the routes are `/v1/discovery/{publish,unpublish,state}`, `/v1/system/status`, - and `/v1/auth/{verify-password,set-password,set-role,delete-user}`. See - `specs/BRAIN_HOST_PROTOCOL.md`. + defines the types; `internal/hostclient/` is the brain-side client). The routes + the brain calls today, from `internal/hostclient/`: + `/v1/auth/{verify-password,set-password,set-role,delete-user}`, + `/v1/users/{username}/home`, + `/v1/identity/{well-known,app-service,app-service/release}`, + `/v1/discovery/{publish,unpublish}`, + `/v1/system/{status,resources,gpu,set-timezone}`, + `/v1/health/system`, `/v1/journal/follow`, and the job routes + `/v1/jobs/system-update` + `/v1/jobs/{id}`. `specs/BRAIN_HOST_PROTOCOL.md` + owns the wire and also lists the routes that are specced but not built yet + (files, network, terminal, drive jobs). ## Inside the brain Packages under `internal/` and what each owns. Layer rules come from [`../CLAUDE.md`](../CLAUDE.md) # Go code discipline; only the directional rules -are stated below. +are stated below. The table is the whole of `internal/`'s top level; the +host-side implementation packages under `internal/hostagent/…` are covered by +the `host-agent-real` row in # Components and by # What is not built yet. | Package | Owns | Imported by | |---|---|---| @@ -92,6 +101,11 @@ are stated below. | `assertion` | Verifies the portal's short-lived Ed25519 ownership assertion for the hosted portal-to-box SSO handshake (`Verify`: signature + expiry; box-id/issuer/replay are the handler's policy). Minimal signed token, not a JWT. Mirrors the cloud signer's wire format. Leaf package. | `api` | | `audit` | Append-only `audit_events` table writes. Every elevation-class mutation calls `audit.Record` on success **and** failure. | `api` | | `events` | In-memory pub-sub bus for SSE. Lifecycle stages publish; the SSE handler subscribes. | `lifecycle`, `api`, `cmd/brain` | +| `health` | The brain's typed-issue registry (`HEALTH.md`). The taxonomy is registered in code — a stable string ID binds severity / category / tier / `blocks_*` at registration, never redeclared per raise. Writes through to SQLite on every raise/clear so issues survive a brain restart. | `api`, `notify`, `store`, `cmd/brain` | +| `notify` | Routing + derivation for the dashboard notification center (`NOTIFICATIONS.md`). Notifications are *derived* from events that already exist — today, health raise/clear transitions through a code-registered allowlist — never a parallel taxonomy. Coalesces by `dedup_key`, and emits the member-transparency variant for box-blocking storage issues. | `api`, `store`, `cmd/brain` | +| `applog` | Per-app log fan-out (`BRAIN_UI_PROTOCOL.md` Pattern C, `LOGGING.md` # Per-app logs). Sits between host-agent's single upstream follow per instance and the dashboard's many SSE readers, and owns the reconnect contract host-agent deliberately does not: a ~256 KiB ring buffer, replay from `Last-Event-ID`, one `{"lost":true}` marker when a position was evicted, and a linger so a quick reconnect reuses the warm buffer. One ref-counted `Hub` per instance — zero idle cost when nobody is watching. | `api`, `cmd/brain` | +| `systemlive` | The live system-resources stream (`BRAIN_UI_PROTOCOL.md` Pattern C stream 3, `LOCAL_ANALYTICS.md`). Ref-counted upstream poller: the first SSE subscriber starts a 1 Hz poll of host-agent's raw cumulative counters, each poll is diffed into rates and fanned out, the last unsubscribe stops it. Same zero-idle-cost shape as `applog`. | `api`, `cmd/brain` | +| `storageverify` | The canary + enrollment-marker check behind the `malmo-storage-verify` reporter (`BOOT.md` # The storage-ready target, `STORAGE.md` # Storage canary). Split out of `cmd/` only so the check is unit-testable against a tempdir root; the binary is a thin shell that writes findings to `/run/malmo/health/storage.json`. **Not a brain package** — it is imported by `cmd/malmo-storage-verify` alone. | `cmd/malmo-storage-verify` | | `version` | The malmo build identity: `Version` (repo `VERSION` file) and `Commit` (git sha), stamped at build time via `-ldflags -X` (`Makefile`, `BUILD.md` # Versioning). Dumb — vars + a `String()`, no logic. | `api`, `hostagent`, `cmd/brain`, `cmd/host-agent`, `cmd/host-agent-real` | **Cross-cutting invariants:** @@ -115,10 +129,10 @@ are stated below. state/ malmo.db brain's SQLite (schema in internal/store) instances/ per-app state (compose file, .env, digests) + services/ managed-service data (postgres-/, valkey-/, …) + catalog-cache/ proxied catalog icons + screenshots (24h expiry) host-agent built binary brain built binary -catalog/ - whoami/ sample manifest (door-1 source) dev/ caddy.json Caddy bootstrap config (replaced live via admin API) docker-compose.yml brings up the dev Caddy + malmo-ingress network @@ -133,16 +147,28 @@ and host-agent agree on paths. brain, Vite — in one terminal. `make help` lists the per-process targets for the four-terminal layout. See [`dev/running-locally.md`](dev/running-locally.md) for the full inner loop. The VM-based outer loop for host-integrated parts -(boot, LUKS, systemd) is not wired here yet. +(boot, LUKS, systemd) is not wired into the native dev loop — it lives in the +QEMU lanes (`specs/TESTING.md`). ## What is **not** built yet So this doc isn't read as a claim about the finished product: - **Full real host-agent.** `cmd/host-agent-real` is partially real: PAM password verify, `/proc` system sampling, disk usage, RAM pressure, journal streaming, service health, reboot-required, discovery (per-LAN-interface Avahi announcements from the NetworkManager LAN set, allowlist sync, IP-change replay), and the first-boot brain launch (#164: load-if-absent, lockstep label check, `docker run --restart unless-stopped`) are wired. LUKS/TPM, apt, and the NM configuration surface (WiFi setup, `/v1/network/*`) are not yet wired — those ops are still no-ops or stubs. -- **Control-plane stack bring-up — built (M1b, #165), VM-boot acceptance pending.** host-agent seeds the brain's Docker transport (the `malmo-ingress` network + the `docker-socket-proxy`, raw socket `:ro`, `EXEC` denied) before launching the brain, and points it at `DOCKER_HOST=tcp://docker-proxy:2375`; the brain then reconciles Caddy + `malmo-ui` from the staged control-plane compose (`lifecycle.EnsureControlPlane`) and installs the dashboard route (`/api/v1/* → brain`, else → `malmo-ui`). All of it is **production-gated** on `MALMO_CONTROL_PLANE_DIR`/`MALMO_DASHBOARD_UI_UPSTREAM`, so the natively-run dev brain is unchanged (standalone dev Caddy, Vite UI, raw socket). Managed-DB-in-production stays gated on a provisioning re-architecture off `docker exec` (the proxy denies `EXEC` — `DECISIONS.md` 2026-06-14). Unit-tested; a real VM boot pass (`sudo make test-medium-qemu`) is still outstanding. -- **Storage subsystem.** No `/srv/malmo`, no mergerfs, no LUKS-unlock flow, - no `malmo-storage-ready.target`. Apps write to wherever Docker puts volumes. +- **Control-plane stack bring-up — built (M1b, #165), VM-boot acceptance pending.** host-agent seeds the brain's Docker transport (the `malmo-ingress` network + the `docker-socket-proxy`, raw socket `:ro`, `EXEC` denied) before launching the brain, and points it at `DOCKER_HOST=tcp://docker-proxy:2375`; the brain then reconciles Caddy + `malmo-ui` from the staged control-plane compose (`lifecycle.EnsureControlPlane`) and installs the dashboard route (`/api/v1/* → brain`, else → `malmo-ui`). All of it is **production-gated** on `MALMO_CONTROL_PLANE_DIR`/`MALMO_DASHBOARD_UI_UPSTREAM`, so the natively-run dev brain is unchanged (standalone dev Caddy, Vite UI, raw socket). Managed DB in production is **no longer blocked**. #185 moved provisioning off `docker exec` onto a one-shot `--rm` client container. So `EXEC` stays denied, and the brain stays off the app-reachable `malmo-svc-*` network (`DECISIONS.md` 2026-06-15, which lifts the 2026-06-14 gate). The bring-up is proven on a booted box by the **hosted** cloud lane, which runs in CI. The **appliance** medium-lane run (`sudo make test-medium-qemu`) is still outstanding. +- **Storage subsystem — the boot half exists, the pooling half does not.** The + userspace boot chain is real and shipped in `dist/systemd/` + (`malmo-storage-ready.target`, `malmo-storage-verify.service`, + `malmo-recovery.target`) with the reporter in `cmd/malmo-storage-verify` and + the health wiring in `internal/health` + ([`progress/boot-pipeline-units.md`](progress/boot-pipeline-units.md)), and + **LUKS root + first-boot TPM enrollment + unseal is proven end to end** in the + QEMU medium lane against a real kernel and a software TPM + ([`progress/luks-tpm-enrollment.md`](progress/luks-tpm-enrollment.md)). What + does **not** exist: the data-drive half — no mergerfs assembly, no `/srv/malmo` + pool (the path appears only as the storage-verify canary), no UI or host-agent + surface for adding a drive or unlocking one. On a dev box apps still write to + wherever Docker puts volumes. - **Boot, install ISO, updates.** The `mkosi` image build (`BUILD.md` # 2; proven in the test lane, not yet the production ISO) and stream A (`unattended-upgrades` + the apt repo) are spec-only. **Stream B — the @@ -178,12 +204,30 @@ So this doc isn't read as a claim about the finished product: box↔cloud credential, `NEXT.md` Tier 1), no update notification, and no dashboard surface beyond Settings → About reporting the running versions (#393). -- **Health / notifications / telemetry / time / discovery beyond stubs.** The - brain doesn't surface health issues, the bell doesn't exist, no telemetry - client, no chrony integration. -- **Login UI.** `Setup` and `Dashboard` render; `Login.vue` is kept in the tree - but not routed (single-user dev phase). Cookie sessions and the underlying - auth pipeline are real. +- **File manager.** `FILES.md` is written, and `/files` is a real top-level + route. Nothing behind it is built. The brain registers no `/api/v1/files/*` + handlers, and host-agent implements none of the `/v1/files/*` ops the protocol + reserves. `FilesView.vue` is a "coming soon" stub. +- **Telemetry — we record consent, and send nothing.** The first-run consent + choice is stored (`POST /api/v1/system/telemetry` → `box_meta`). Nothing goes + anywhere after that: there is no telemetry client, and no endpoint to send to + (`TELEMETRY.md`). Health, notifications, time and discovery used to sit in + this bullet too. They are built now: `internal/health` raises typed issues + that show on `GET /api/v1/health` and in the dashboard's `HealthBanner`; the + notification centre and its bell work (`internal/notify`, + `NotificationBell.vue`, mute per category); the `clock-not-synced` detector + reads real `chronyc tracking` (`internal/hostagent/clockhealth`); the time + zone can be set through host-agent; and Avahi discovery is listed as real in + # Components above. +- **Off-box notification transports.** The bell is dashboard-only. No email and + no push, so a box that needs attention while nobody is looking at the + dashboard cannot say so (`NOTIFICATIONS.md`, `specs/NEXT.md` Tier 2). +- **Login UI — it renders, but it is not a route.** `Login.vue` is real and + users reach it. `App.vue` chooses between `Setup`, `Login` and the dashboard + based on auth state, so a logged-out appliance box shows the login screen with + no route change. There is no `/login` route: the logged-out screens are app + state, not router entries. `/recover` is the one exception. Cookie sessions + and the auth pipeline behind the screen are real. - **App store.** Every box syncs the catalog from the control plane (`GET /catalog/sync`) with integrity-digest verification and TLS for authenticity; no catalog is baked into the image, the box keeps no copy on disk (`DECISIONS.md` 2026-08-17), and there is no Ed25519 signature (`DECISIONS.md` 2026-07-02). What remains is cloud-side: the store is the authoring surface. The door-1 app-authoring how-to (`docs/dev/authoring-apps-with-an-agent.md`) is reconciled with that — it authors into `store:apps//` and keeps the schema, tooling, and gap ledger here. For where each of these is planned, see the matching `specs/` doc. diff --git a/docs/dev/catalog-import-gaps.md b/docs/dev/catalog-import-gaps.md index 2901072..d9167d8 100644 --- a/docs/dev/catalog-import-gaps.md +++ b/docs/dev/catalog-import-gaps.md @@ -263,3 +263,19 @@ Unlike `docs/progress/` entries (frozen ADR snapshots), this file is **mutable b - **What breaks:** neither credential set can be marked required, because only one is ever filled and requiring both would make install impossible. So the manifest ships all five fields optional and the toggle/field pairing survives only as prose in the field descriptions and the store description. A user who flips a toggle on and saves without filling that method's fields gets a **non-booting app**: Flipt's config validation rejects an enabled method with an empty `client_id` and exits before serving (`internal/config/authentication.go`, `AuthenticationMethod.validate()`). Recoverable — clearing the toggle from the app's config page restarts it cleanly — but the platform had every fact needed to refuse the bad save and instead let it through to a crash. Recorded on the app side as `apps/flipt/` limitation `auth-toggle-without-credentials` (`degraded`/`open`). - **Why malmo can't satisfy it (v1):** `ConfigField.Required` is a plain `bool` on a flat field list (`internal/manifest/manifest.go`), evaluated per-field with no reference to any other field's value. There is no dependency expression (`required_when: == true`), no field grouping, and no cross-field validation hook at either of `config:`'s two entry points (the install consent form and the post-install Settings editor), so the brain cannot know that one blank field is fine and another is fatal. The shape isn't Flipt-specific: it recurs for any app offering **alternative** credential sets rather than a flat list of independent optional keys — pick-one-of-N auth providers, pick-one storage backend (S3 keys vs local path vs Azure), pick-one mail transport. Postiz was the easy case that hid this — its provider keys are genuinely independent and individually optional, so a flat list expressed it exactly; a blank key there just leaves one network disabled instead of killing the boot. - **Status:** open — shipped in `apps/flipt/` (store repo) with all five fields optional and the pairing documented in prose; the app ships `state: degraded` partly for this. Worth noting for triage that the cheap fix may not be `required_when` at all: a `group:` + `one_of` construct (declare the two credential sets as alternatives, brain enforces "exactly one group complete") covers the same apps and reads better on the install form than per-field dependency expressions. Second app in this shape closes the argument either way; one app is not yet a pattern. + +### grpc-ingress — laminar (2026-08-14) + +- **Severity:** degraded +- **Trigger:** Laminar's `app-server` listens twice — HTTP on `PORT` (8000) and **gRPC on `GRPC_PORT`** (8001) — and its SDKs export OpenTelemetry spans over gRPC **by default**. A manifest names exactly one `main_service` + `main_port`, so only one of the two can be routed. +- **What breaks:** an SDK left on its defaults sends to `:8443` and nothing arrives. Verified on a real install: the Python SDK retried against `laminar.local:8443` with `StatusCode.UNAVAILABLE … Domain name not found` and no span reached the app; with `force_http=True` the same run posted to `/v1/traces` and the trace was readable seconds later. The JS SDK is worse — the store issue records it as failing quietly. +- **Why malmo can't satisfy it (v1):** one app gets one routed port, and the ingress is an HTTP reverse proxy. Even given a second port, the box's Caddy dials **plain HTTP** upstreams, so h2c (gRPC's cleartext HTTP/2) would not survive the hop; gRPC needs either end-to-end HTTP/2 or a `grpc_pass`-style upstream mode, and neither exists. The app-side proxy pattern this repo uses for multi-listener apps (langfuse, plane, baserow, and laminar itself) solves *path* fan-out on one HTTP origin and cannot help here, because the problem is protocol, not path. +- **Status:** open — shipped `degraded` (`malmoos/store` `apps/laminar/`, limitation `grpc-export-not-routed`), with the HTTP-export workaround named in the app's own store description because an unconfigured SDK fails silently. First app of this gap-class. Expect recurrence: any app whose SDK speaks gRPC to its own backend (OTel collectors and agent frameworks are the likely repeat offenders). + +### self-call-to-app-url — laminar (2026-08-14) + +- **Severity:** blocks-use (see note) — the app boots, serves, and signs you in, but you cannot open a project +- **Trigger:** `NEXTAUTH_URL` — `frontend/middleware.ts` authorizes every `/project/{id}` page and every `/api/projects/{id}/…` route by `fetch`ing **its own public URL** from inside the container (`${NEXTAUTH_URL}/api/auth`, with a shared-secret bearer). The app is asking to reach itself at the address the *browser* uses. +- **What breaks:** the fetch cannot complete, and the middleware treats any failure as "not authorized" (`if (!res.ok) return false`, plus a `catch` that also returns false). So there is no error anywhere: every project page **redirects to `/sign-in`**. You sign in, see your workspace, click a project, and land back at sign-in. Measured on the first boot: `fetch failed` from inside the container, `307` on `POST /api/projects//api-keys`. It fails on both profiles for different reasons — on **appliance** the container cannot resolve the box's `.local` mDNS name; on **hosted** the name resolves but the request re-enters through the box's ingress and meets the owner-only forward-auth gate, so the middleware gets a login redirect instead of its answer. +- **Why malmo can't satisfy it (v1):** the brain injects `MALMO_APP_URL` and nothing else about the app's own address (`internal/lifecycle/lifecycle.go` writeEnv) — no in-network equivalent, and no bare-host variant (the same missing `MALMO_APP_HOST` the `app-url-injection — docuseal` entry names). Nothing resolves the app's public name inside its own compose network: appliance `.local` names are published to the LAN by Avahi, not to container DNS, and on hosted the public name resolves only to the box's ingress, which is exactly the thing that gates the request. `access.public_paths` (#415) can carve the called path out of the gate on hosted, but that turns an internal authorization check into an internet round trip on every page load, and does nothing on appliance. +- **Status:** open — laminar shipped by **routing around it**: `NEXTAUTH_URL` is set to the compose-network address (`http://frontend:5667`) so the self-call is local, and `NEXT_PUBLIC_URL` keeps `${MALMO_APP_URL}` for the links the UI renders. That works only because next-auth v4 builds its post-sign-in redirect from the request rather than from `NEXTAUTH_URL` (verified: sign-in still returns the public origin). The gap stays open because the workaround is app-specific luck: an app that uses one variable for **both** the browser-facing origin and its own self-call, or that hardcodes the public URL for one, has no such split. The plausible mechanisms are a `MALMO_APP_INTERNAL_URL` injection (compose-network address of `main_service`) or in-network DNS for the app's own routed name; neither exists. Severity is deliberately recorded as `blocks-use` rather than `blocks-start` — this ledger's severity list predates that state, and "boots fine, cannot be used" is the honest reading (the store's own severity vocabulary added `blocks-use` for exactly this shape). diff --git a/docs/dev/contributing.md b/docs/dev/contributing.md index 172cc66..f7a4a7d 100644 --- a/docs/dev/contributing.md +++ b/docs/dev/contributing.md @@ -8,7 +8,7 @@ This guide is written so a contributor **and their coding agent** can both follo Read in this order. Don't skip [`../../CLAUDE.md`](../../CLAUDE.md) — it holds the load-bearing conventions and overrides default agent behavior. -1. **[`../../CLAUDE.md`](../../CLAUDE.md)** — what malmo is, the audience, the locked decisions, and the code/doc discipline you're held to. The **Documents** section is the annotated map of every spec. +1. **[`../../CLAUDE.md`](../../CLAUDE.md)** — what malmo is, the audience, the locked decisions, and the code/doc discipline you're held to. The annotated map of every spec lives in [`../README.md`](../README.md) # Specs. 2. **[`../specs/SPEC.md`](../specs/SPEC.md)** and **[`../specs/CONTROL_PLANE.md`](../specs/CONTROL_PLANE.md)** — the vision and the control-plane architecture (brain + host-agent + Caddy). 3. **[`../README.md`](../README.md)** — the doc map. You don't read every spec now; you read the one(s) your task touches, end-to-end, when you pick it up. 4. **[`running-locally.md`](running-locally.md)** — get the stack running natively (no VM) before you write a line. diff --git a/docs/dev/hosted-boot-proof.md b/docs/dev/hosted-boot-proof.md index 1e535dd..583a8f6 100644 --- a/docs/dev/hosted-boot-proof.md +++ b/docs/dev/hosted-boot-proof.md @@ -72,7 +72,7 @@ A **broken image build** presents as several of these at once (e.g. `:443` refus ## How to run it - **It can also start on its own.** A PR into `dev` or `main` that touches the updater — `internal/hostagent/**`, `cmd/host-agent-real/**`, `internal/lifecycle/controlplane*.go` or `lifecycle.go`, `internal/api/system*.go`, `dev/cloud/**`, or the workflow itself — runs `CI / Cloud image` automatically and boots **only** the `update` scenario (#389). It publishes nothing (the run asserts that before it builds). So a PR in those paths can turn this lane red without anyone asking for it; read the run the same way as a manual one. Expect ~12–15 min, most of it the image build. Draft PRs are skipped — marking the PR ready starts the run. A PR that needs the other boots can still run them by hand with the command below. -- **CI (preferred — no local root/KVM, no image push):** `gh workflow run "CI / Cloud image" --ref -f publish=false`. Builds the image, then runs the `unseeded seeded bios access update` boots under QEMU. `publish=true` (the default) additionally uploads the built image to the provider — only do that deliberately. Runtime ~10 min for the build, plus the boots (the `update` boot is the longest — it runs two full update transactions, one of which spends a 60s health wait failing on purpose). +- **CI (preferred — no local root/KVM, no image push):** `gh workflow run "CI / Cloud image" --ref -f publish=false`. Builds the image, then runs the `unseeded seeded bios access update` boots under QEMU. `publish=true` (the default) additionally **attaches the compressed image + checksum to the tagged GitHub Release and pushes the brain + UI images to ghcr** (#352 removed the old provider-snapshot upload, so the lane holds no hosting-provider credential) — only do that deliberately. Runtime ~10 min for the build, plus the boots (the `update` boot is the longest — it runs two full update transactions, one of which spends a 60s health wait failing on purpose). - **Local:** `sudo make test-cloud-qemu` (needs root + `/dev/kvm`). Scope boots with `MALMO_CLOUD_BOOTS="seeded"` to reproduce the wildcard path alone, `"bios"` for the legacy-BIOS boot alone, `"update"` for the updater proof alone, or the default `"unseeded seeded frozen bios access update"` for the full run. ## Related history (frozen snapshots — background, not the current view) diff --git a/docs/progress/README.md b/docs/progress/README.md index 84c3bf9..fc2cf55 100644 --- a/docs/progress/README.md +++ b/docs/progress/README.md @@ -166,7 +166,9 @@ Oldest first; append new entries to the bottom. | [hosted-cloud-image.md](hosted-cloud-image.md) — Lean hosted cloud-image mkosi profile + `hosted` marker (C1b, **closes #203** / part of #196; after #202/C1a marker, alongside #204/C1c slim agent, before #205/C2 boot): the repo's first *product* image definition. New `dev/cloud/` mkosi tree — its **own** image (not the appliance rootfs with services disabled, `ENVIRONMENT.md` "absent, not disabled"), `Distribution=debian`/`Release=trixie` (locked product base, not the test lane's bookworm), `Format=disk`, `ImageId=malmo-cloud`, `ManifestFormat=json`, `ToolsTree=default`, `Bootable` + `systemd-boot`, `KernelCommandLine` carrying **`psi=1`** (required so the `ram-pressure` detector reads real PSI — the test lane omits it) and a virtio+ext4+crc32c modules-initrd (the LUKS/dm-crypt modules deliberately absent). **Lean package set** (`linux-image-amd64`, systemd stack, `docker-ce` stack, `ca-certificates`, `dbus`, `util-linux`, `kmod`, `udev`, `iproute2`, `sudo`, base shell utils); the appliance cuts — `network-manager`, `avahi-daemon`/`avahi-utils`, `samba`, `mergerfs`, `cryptsetup`, `tpm2-tools`, `openssh-server` — **not installed**. **No LUKS/TPM repart** (ships no `mkosi.repart/` → mkosi-default plain ESP+root). Marker `dev/cloud/mkosi.extra/etc/malmo/profile`=`hosted` baked via `ExtraTrees` (a static committed file — exactly what `internal/profile.Read`/#202 reads). `dev/cloud/bootstrap.sh` (thin analog of the test lane's): preflight → stage Docker apt repo (trixie pocket) → `mkosi build` → **assert the cut list absent from the JSON manifest (exact-name match, so `libcryptsetup12` is not a false positive) + marker = `hosted`**. `make build-cloud-image` wired. **Firewall posture decided, not omitted** (`DECISIONS.md` 2026-06-19): hosted relies on the **cloud provider's security groups** (explicit operator requirement; the appliance `nftables` only LAN-scopes SSH/SMB, both dropped) — in-guest `nftables` backstop deferred (`NEXT.md`); `ENVIRONMENT.md` # Public-by-default made explicit. **Config validated** via `mkosi summary` under mkosi 26 (main image resolves trixie/disk/json/`Repart Directories: none`/lean packages); **full `mkosi build` not exercised here** — blocked by the mkosi-26/Ubuntu-24.04 `PR_CAPBSET_DROP` sandbox limitation (#189), same as the medium QEMU lane. Boot proof + qcow2 + control-plane bring-up are #205/C2 | done | | [hosted-profile-marker.md](hosted-profile-marker.md) — Environment-profile marker + brain reads it (**C1a of #196**, the cloud-VM track): the foundational, mkosi-independent seam letting ~95% of the brain (Layer 1) stay identical across the `appliance` and `hosted` profiles. New `internal/profile` provider package (concrete types only) exports `Profile` (`Appliance`\|`Hosted`), `DefaultMarkerPath` (`/etc/malmo/profile`), and `Read(path)` — which never fails: an absent marker (unmarked appliance box / `make dev`), an unreadable one, or unrecognized content all resolve to `Appliance` (the no-op default, so an unmarked box is unchanged); surrounding whitespace/trailing newline tolerated. Logging is asymmetric — absent is the silent expected case, present-but-unreadable and present-but-unrecognized both `slog.Warn` (a mis-stamped image shouldn't silently run in the wrong posture). `cmd/brain/main.go` reads it **once at startup** (env `MALMO_PROFILE_PATH`, default the marker path) into a local `prof` logged via the new `profile` structured field (added to `CLAUDE.md` # Go code discipline); `prof` is held for future seams but consumed by none yet — no image/host-agent/behavioral-seam change here (those are C1b #203 / C1c #204). Value-matrix tests at 100% coverage; `make check` green (`ENVIRONMENT.md` # How the profile is realized) | done | | [slim-cloud-host-agent.md](slim-cloud-host-agent.md) — Build-tagged slim cloud `host-agent` (**closes #204**/C1c; last of the C1 split after #202/C1a + alongside #203/C1b, part of #196). `cmd/host-agent-real` now builds in two profiles from one tree via the `hosted` Go build tag: `main.go` boilerplate (socket bind/serve, first-boot brain bootstrap, env helpers) stays shared/untagged, and the host-op **seam composition** splits into `wiring_appliance.go` (`//go:build !hosted`, today's full wiring byte-for-behavior unchanged) and `wiring_hosted.go` (`//go:build hosted`, the slim cloud set). Hosted **keeps** real PAM verify, `usermgr`, the storage/services/time/resources/reboot/system + disk + per-app-log reporters (none touch the LAN/NM/DBus); **drops** `avahipublisher` (mDNS publish) and `netstate.NMProvider` + the NetworkManager watcher; LUKS/TPM, Samba, nftables aren't in the hosted image (#203) so there's nothing to wire. Two no-ops keep the shared mux safe: a 3-line `noopPublisher` (the brain already skips `POST /v1/discovery/publish` in `hosted` via the C1a marker — belt-and-suspenders) and `a.Net` left nil (`GET /v1/discovery/state` already reads nil as "not measured"; net-state seam choice = nil stub, **not** a kernel reader, to avoid pulling NM). Both profiles stay Linux + CGO + libpam0g-dev (PAM kept). `make host-agent-real-hosted` (`go build -tags hosted`) builds the cloud binary; #203/C1b + #205/C2 consume it. Verified slimmer: **`avahipublisher` absent** from the hosted import graph, godbus symbols 378→182, ~410 KB smaller; default appliance build unchanged (`make check` green); vet/build green both with and without `-tags hosted` across the whole module (no ripple). **Known residual:** `netstate` stays imported (godbus rides along) because the shared `hostagent.NetState` seam returns `netstate.LANInterface` and #204 forbids changing the seam / scopes to consumer-side wiring — inert (`a.Net` nil), filed as #217. No VM-boot proof here (that's **#205/C2**, which bakes + boots the image) | done | +| [cloud-vm-boot-proof.md](cloud-vm-boot-proof.md) — Hosted cloud VM image: emit + boot the control plane (**C2 of #196**, closes #205; after C1a #202 / C1b #203 / C1c #204 and C3a #206). The first *real* build of the lean cloud image: it is emitted, converted to the `malmo-vX.Y.Z-amd64.qcow2` cloud artifact (`BUILD.md` # 6), and **booted once in QEMU under OVMF UEFI** — no swtpm, no LUKS, no installer (`ENVIRONMENT.md` # Provisioning: "the disk IS the installed system"). New `dev/cloud/test/` boot-proof image `Include=`s the lean `dev/cloud/mkosi.conf` rather than copying it, so what boots is the lean image plus instrumentation, never a divergent rebuild; `dev/cloud/run-cloud-tests.sh` boots it air-gapped (`restrict=on`, so a stray pull hard-fails and the offline bundle is proven complete) with `snapshot=on` and serial capture; a first-boot oneshot writes `MALMO_CLOUD_ASSERTIONS: PASS|FAIL` to the **serial console** (hosted ships no sshd) and the driver greps it. Assertions: userspace converged with no failed control-plane unit, PSI live (pairs with C1b's `psi=1`), `systemd-networkd` active + NetworkManager absent, docker active with the four baked images loaded, the four control-plane containers running, the proxy boundary (brain has no raw `docker.sock`), the dashboard answering through Caddy, and the hosted `/setup` gate returning **503**. **Integration bug this caught:** the brain container mounts only the agent socket dir + `/var/lib/malmo`, neither covering `/etc/malmo` — so `profile.Read` inside the container always returned `Appliance` and hosted `/setup` would have run **open**, the exact public-by-default hole C3a's gate exists to close (C3a's unit tests override `MALMO_PROFILE_PATH`, so they could not see it). Fixed host-agent-side: `brainlaunch.Config.ProfileMarkerPath` bind-mounts the marker read-only at the same path, set only when it exists as a regular file. Two lean-package gaps also surfaced on the first real build (`passwd`, `systemd-boot-efi` — both arrived transitively via packages the hosted image cuts). `make test-cloud-qemu` wired. Seed delivery (C3), the trimmed wizard (C4) and the end-to-end (C5) stay out of scope — C2 asserts the brain *serves*, pre-setup | done | | [hosted-setup-gate.md](hosted-setup-gate.md) — Hosted first-boot seed + admin-bootstrap gate on `/setup` (**C3a of #196**, brain-side core only): a hosted box has no "whoever is physically present at first boot" trust, so the first admin's creation is gated on a one-time secret from the provisioning seed (`ENVIRONMENT.md` # Provisioning). New `internal/profile/seed.go` (`Seed{box_id, admin_bootstrap_secret, enrollment json.RawMessage}` from `/var/lib/malmo/seed.json`, override `MALMO_SEED_PATH`; `ReadSeed`→ typed `ErrSeedAbsent` on missing, hard error on malformed/incomplete; `enrollment` reserved-unconsumed for C3b). New `box_meta` KV in `internal/store` persists the box-id + the **SHA-256 hex of the secret, never plaintext**. `cmd/brain` `loadHostedEnvironment` ingests once at startup — appliance no-op; hosted first boot reads the seed and persists **hash then box-id** (box-id last = crash-safe commit marker); a persisted box-id is the install's **frozen identity** (seed ignored on later boots); absent/unreadable seed stays pre-setup (never falls back to appliance open). `internal/api` `gateBootstrap` runs in `/setup` before the empty-box check (hosted only): no seed → **503**, missing/wrong `bootstrap_secret` body field → **401** audited `setup.failure`, correct → proceeds; **constant-time** (`crypto/subtle`) compare over the hash; naturally one-time via the existing empty-box guard. `UserDTO.box_id,omitempty` surfaced on hosted only; **appliance `/setup` byte-unchanged**. OpenAPI + web-ui client regenerated (only `bootstrap_secret` + `box_id` added). Cloud-lane seed delivery (SMBIOS/QEMU) deferred (depends on C2 #205, #189-blocked); PR is "Part of #206", not "Closes". `make check` green (`ENVIRONMENT.md` # Provisioning & first-boot) | done | +| [cloud-seed-delivery.md](cloud-seed-delivery.md) — Hosted first-boot seed delivery + `/setup` gate end-to-end (**closes #220**; completes the deferred half of C3a #206, [hosted-setup-gate.md](hosted-setup-gate.md), on the C2 harness [cloud-vm-boot-proof.md](cloud-vm-boot-proof.md)). C3a built the brain-side seed ingestion and the admin-bootstrap gate and unit-tested it, but nothing ever delivered a `seed.json` onto a booted VM, so the gate had never run end-to-end. **No brain/Go change** — the brain already reads `/var/lib/malmo/seed.json` on `hosted`, and `brainlaunch.runSpec` already binds `/var/lib/malmo` into the brain container, so a seed written to that host path before host-agent launches the brain is what the containerized brain reads. New `dev/cloud/test/malmo-seed.service` (first-boot oneshot, `Before=host-agent.service`, `ImportCredential=malmo.seed`) + `malmo-seed-materialize.sh` (`install -m 0600 root:root`; no-op when nothing was delivered, so an un-seeded boot stays unprovisioned). `run-cloud-tests.sh` refactored into a `run_boot [seed-cred…]` helper driving a **3-boot sequence over one persisted qcow2 overlay** (box-id + first admin survive boot→boot; the base artifact stays pristine), the seed delivered as an SMBIOS type-11 systemd credential (binary/base64 so the JSON is comma- and newline-safe). The three boots: un-seeded ⇒ `/setup` 503; seeded ⇒ wrong secret 401 + correct secret 200 with `box_id=cindy-fox`; frozen-identity reboot ⇒ a re-delivered *different* seed (`rusty-hawk`) is ignored and `/login` still reports `cindy-fox` (the materializer deliberately overwrites, so that assertion is non-vacuous — the identity lives in SQLite, not the file). Note the delivery channel is a **QEMU-lane** mechanism: a real box is seeded once as metadata user-data at VM creation (`ENVIRONMENT.md` # Provisioning & first-boot) | done | | [sandbox-one-uid-per-app-triage.md](sandbox-one-uid-per-app-triage.md) — Triage of the one-UID-per-app sandbox limitation (**#193**, a design/triage item filed off Formbricks #182). malmo pins one malmo-assigned UID + `cap_drop: ALL` + `no-new-privileges` on **every** service and rejects a numeric `user:` at admission; that breaks images that mutate their own image-baked files owned by a fixed non-root UID (Formbricks' `nextjs`/1001 migration runner — root under `cap_drop: ALL` can't write them either, no `CAP_DAC_OVERRIDE`). Verdict: **no code change.** The directions #193 floated (a manifest field naming a UID / "run as the image's declared `USER`") are foreclosed by a locked decision — `APP_ISOLATION.md` # Runtime identity forbids a manifest-named UID (host-namespace aliasing, `THREAT_MODEL.md`; `DECISIONS.md` 2026-06-10). Both Formbricks cases reduce to the **already-deferred hardcoded-internal-UID class** (same as poznote/postiz): case 2's "conflicting per-service UID" adds nothing — a bundled Postgres adopts any non-root UID via `service_user`, so only the main app's hardcoded 1001 blocks (case 1), and per-service distinct UIDs wouldn't help. Early detection is already covered by the mandatory curation boot smoke test. Recorded the triage in the doc homes: new `nonroot-data-ownership — formbricks` ledger entry, updated `NEXT.md` # User-namespace remap item (count-trigger now met but gated on its feasibility spike; per-service-UID note added), one-sentence tightening of `APP_ISOLATION.md` # What this does not cover. Docs-only PR; Formbricks stays a curation-reject pending userns-remap | done | | [hosted-first-run-wizard.md](hosted-first-run-wizard.md) — Trimmed hosted first-run wizard (**C4 of #196**, closes #208): the web-ui half consuming C3a's `/setup` gate, plus the brain endpoints its non-account steps need. New host seam `POST /v1/system/set-timezone` (`protocol.SetTimezoneRequest` → `hostclient.SetTimezone` → `hostagent.TimezoneSetter`, **nil ⇒ accepted no-op 200** for the fake/dev loop) backed by new pkg `internal/hostagent/timezone` (real `timedatectl set-timezone`, zone-shape regex guard; wired in **both** `host-agent-real` build profiles). New `internal/api/firstrun.go` — admin-gated `POST /system/{timezone,telemetry,first-run-complete}` (timezone validates IANA shape→422 then pass-through→502; telemetry + first-run-complete persist to C3a's `box_meta` via new keys `telemetry_consent`/`first_run_complete`, no migration). `GET /auth/state` extended to `{has_users, first_run_complete, profile}`: the dashboard gates the wizard on **`first_run_complete`** (not `has_users`) so a half-finished wizard resumes, and `profile` selects the step set + bootstrap-secret field. `POST /setup` gained optional `recovery` (`nil`/`true`→code as before; `false`→no hash + empty `recovery_code`, recover-flow unredeemable). web-ui: `Setup.vue` rewritten as a **data-driven profile-aware shell** (steps `[admin, timezone, telemetry, done]`; B4 splices network/enrollment per profile) + `web-ui/src/setup/{Admin,Timezone,Telemetry,Done}Step.vue`; `App.vue` renders the wizard while `!firstRunComplete` with a `!firstRunComplete && hasUsers && !currentUser → Login` resume-reauth branch. Telemetry copy verbatim from `TELEMETRY.md` (incl. PostHog disclosure); recovery toggle per `FIRST_RUN.md` # Step 2a. Deviations: time-zone detect uses browser `Intl` zone (not IP-geo); admin step collects a Linux username (display-name→slug **deferred**); `timedatectl` exec untested (host-integration lane, `usermgr` convention). OpenAPI + web client regenerated. `make check` + `make check-web` green (`FIRST_RUN.md` # Phase 2, `ENVIRONMENT.md` # Provisioning) | done | | [hosted-wildcard-cert.md](hosted-wildcard-cert.md) — Hosted always-on `.malmo.network` wildcard cert + sole URL scheme (**C3b of #196**, **#207**; consumes the seam C3a/#206 left open, unblocked by cloud CL4/CL5). Box-side machinery for the hosted profile's `..malmo.network` HTTPS scheme. **Typed enrollment:** `profile.Seed.Enrollment` goes from opaque `json.RawMessage` to `EnrollmentCredentials{subdomain,username,password}` matching the cloud producer's wire format byte-for-byte; `loadHostedEnvironment` persists it (`box_meta`, hash→enrollment→box-id commit order) + reloads on frozen-identity boots so Caddy reconfigures without re-reading the ignored seed; absent/incomplete enrollment still gates `/setup`, just serves no HTTPS. **Caddy DNS-01:** new `caddy.EnsureWildcardTLS` writes the `tls` automation policy (ACME issuer, `acmedns` DNS provider with the seeded creds + a box-side endpoint constant `MALMO_ACMEDNS_ENDPOINT`=`https://auth.malmo.network`) and adds `:443`; cert subjects are `.malmo.network` **+** `*..malmo.network` (the apex isn't under the wildcard) sharing one `_acme-challenge.` record; called at startup only when `hosted && enrollment.Complete()`. **Sole scheme:** new `internal/profile/appurl.go` host helpers (the single place the shape is named) consumed by a profile-aware seam in `lifecycle` (`SetEnvironment`/`m.hosted()`: install/`routeHost`/`publishHost`/`MALMO_APP_URL` use the public host, skip mDNS publish) and `api` (`toDTO` → `https://..malmo.network`); dashboard host becomes `.malmo.network`; **appliance `.local`/Avahi path byte-for-byte unchanged**, ready for the future toggle to reuse the seam. **Custom Caddy:** stock `caddy:2-alpine` has no DNS module, so `dev/control-plane/caddy-acmedns/Dockerfile` (xcaddy + `caddy-dns/acmedns`) + `compose.yml` image `${MALMO_CADDY_IMAGE:-caddy:2-alpine}`; hosted sets the var, appliance keeps stock. Unit-tested (config-gen, URL surfacing, enrollment ingest/reload, cert subjects); `make check` green minus PAM (pre-existing CGo). **Real Let's Encrypt issuance NOT verified here** — joint with cloud #6 / CL6 (no real ACME/DNS in the inner loop or QEMU cloud lane); `DECISIONS.md` entries for the hosted custom image + acme-dns endpoint constant; public acme-dns API face not yet deployed cloud-side (filed) | done | diff --git a/docs/specs/APP_ISOLATION.md b/docs/specs/APP_ISOLATION.md index 9988981..adb1df7 100644 --- a/docs/specs/APP_ISOLATION.md +++ b/docs/specs/APP_ISOLATION.md @@ -117,7 +117,7 @@ The writable layer resets on container *recreation* (image update, uninstall/rei ### Volumes -App state (indexes, configs, the app's own DB) lives under the instance dir at `/var/lib/malmo/instances//data/`, via bind mounts only — no Docker named volumes (`APP_LIFECYCLE.md` # on-disk layout per instance). The author writes the bind mount against `${MALMO_DATA_DIR}/foo:/foo` (or the relative `./data/foo:/foo`); the brain injects `MALMO_DATA_DIR`, so authors reference a stable variable rather than a hardcoded host path. +App state (indexes, configs, the app's own DB) lives under the instance dir at `/var/lib/malmo/state/instances//data/`, via bind mounts only — no Docker named volumes (`APP_LIFECYCLE.md` # on-disk layout per instance). The author writes the bind mount against `${MALMO_DATA_DIR}/foo:/foo` (or the relative `./data/foo:/foo`); the brain injects `MALMO_DATA_DIR`, so authors reference a stable variable rather than a hardcoded host path. `/tmp` is a size-capped tmpfs. @@ -320,7 +320,7 @@ Mounted-file secrets (Docker secrets style) deferred — only ~half of images su When a per-user app declares `services: [postgres]`, the brain runs a Postgres container **co-located on that (user, app)'s per-app network**. Only that specific instance — Andrei's Photos, not Maria's — can reach it. - Lifecycle tied to the (user, app) tuple. Uninstall Andrei's Photos → Andrei's Postgres goes away; data backed up first per `SERVICE_PROVISIONING.md`. Maria's Photos is untouched. -- Postgres data lives under the per-(user, app) instance dir (`/var/lib/malmo/instances//managed/postgres/...`), owned by the user's UID with restrictive perms. Cross-user filesystem access is blocked the same way every other app-state dir is — POSIX ownership + the brain controlling the bind-mount surface. **Open:** when fscrypt lands for `/home//`, does it extend to `/var/lib/malmo/instances/` for per-user app state? Tracked in `NEXT.md`. +- Postgres data lives under the per-(user, app) instance dir (`/var/lib/malmo/state/instances//managed/postgres/...`), owned by the user's UID with restrictive perms. Cross-user filesystem access is blocked the same way every other app-state dir is — POSIX ownership + the brain controlling the bind-mount surface. **Open:** when fscrypt lands for `/home//`, does it extend to `/var/lib/malmo/state/instances/` for per-user app state? Tracked in `NEXT.md`. - Network-layer isolation: cross-user, cross-app database access is impossible by construction. - Cost: in the worst case, N users × M apps requesting Postgres = N×M Postgres instances. Realistic case (1–2 users, one heavy account running most apps) keeps this well within home-server budgets. diff --git a/docs/specs/APP_LIFECYCLE.md b/docs/specs/APP_LIFECYCLE.md index 186c12b..d13ddb7 100644 --- a/docs/specs/APP_LIFECYCLE.md +++ b/docs/specs/APP_LIFECYCLE.md @@ -30,7 +30,7 @@ The Docker HTTP API (via the socket proxy) is used directly only for things the ## Locked: on-disk layout per instance ``` -/var/lib/malmo/instances// +/var/lib/malmo/state/instances// ├── manifest.yml # author's, verbatim ├── compose.yml # author's, verbatim ├── compose.override.yml # malmo-generated, regenerated on update diff --git a/docs/specs/APP_MANIFEST.md b/docs/specs/APP_MANIFEST.md index 6196e3c..f97c337 100644 --- a/docs/specs/APP_MANIFEST.md +++ b/docs/specs/APP_MANIFEST.md @@ -172,7 +172,7 @@ Two distinct kinds of storage in every app — user content and app state. See ` **User content** is what the user owns — photos, music, notes, documents. Lives at `/home//Photos/`, `~/Music/`, etc. Apps reach it by **bind-mounting use-case folders**, declared in `permissions.folders` (next section). Survives app uninstall. -**App state** is the app's own working data — indexes, caches, databases, configs. Lives at `/var/lib/malmo/instances//data/`. Opaque to the user. Deleted on uninstall (or archived if the user picks "keep data"). +**App state** is the app's own working data — indexes, caches, databases, configs. Lives at `/var/lib/malmo/state/instances//data/`. Opaque to the user. Deleted on uninstall (or archived if the user picks "keep data"). The `storage:` block configures app state only. @@ -191,7 +191,7 @@ storage: **`data_volumes` vs `cache_volumes`** — the backup system uses this. Cache is regeneratable; data isn't. Without this distinction, we'd back up thumbnail caches. A `cache_volumes` path **may be nested inside a `data_volume`** — the common shape is a single `./data` bind with a `cache/` subdirectory (e.g. a downloaded model or an embeddings store the app re-fetches on demand): list `./data` under `data_volumes` and `./data/cache` under `cache_volumes`, and backup is the data tree **minus** the cache subtrees. Paths that don't nest must not otherwise overlap. (Note both lists are author-grade declarations: v1 parses neither into the Go struct yet — see `docs/dev/authoring-apps-with-an-agent.md` — so they document intent for the backup system that will consume them.) -**`estimated_size` is the *app-state baseline at install* — measured, not a usage projection.** It is the size of the app's own working data (indexes, databases, configs) under `/var/lib/malmo/instances//data/` **as it stands the moment install completes** (the main service first reports healthy), on a clean install. It is deliberately *not* a guess at how big the app might grow with use: if the app later downloads another model or the user uploads a 2 GB library, that growth is **not** counted here — that's a runtime disk-pressure concern (`HEALTH.md` # `disk-full`), not a pre-install figure. The goal is a number close to the real on-disk cost of *having installed* the app; undercounting (a first-boot download still in flight when the health probe passes) is acceptable, overcounting by speculating about use is not. It is **not** the container-image size and **not** the user's content (Photos/Music/Documents the app bind-mounts — that is first-class, unbounded, and survives uninstall, so it is never attributed to the app). Image size is **not** author-declared: the catalog build resolves it from the actual pinned images at publish time (`APP_STORE.md` # Catalog schema). The brain combines the two — image size + `estimated_size` — into the **on-disk footprint** it shows before install (store card + consent dialog; `BRAIN_UI_PROTOCOL.md` # GET /api/v1/catalog/:id/install-plan, `DASHBOARD.md` # Install authorization). It stays advisory: warn on a tight disk, never block. +**`estimated_size` is the *app-state baseline at install* — measured, not a usage projection.** It is the size of the app's own working data (indexes, databases, configs) under `/var/lib/malmo/state/instances//data/` **as it stands the moment install completes** (the main service first reports healthy), on a clean install. It is deliberately *not* a guess at how big the app might grow with use: if the app later downloads another model or the user uploads a 2 GB library, that growth is **not** counted here — that's a runtime disk-pressure concern (`HEALTH.md` # `disk-full`), not a pre-install figure. The goal is a number close to the real on-disk cost of *having installed* the app; undercounting (a first-boot download still in flight when the health probe passes) is acceptable, overcounting by speculating about use is not. It is **not** the container-image size and **not** the user's content (Photos/Music/Documents the app bind-mounts — that is first-class, unbounded, and survives uninstall, so it is never attributed to the app). Image size is **not** author-declared: the catalog build resolves it from the actual pinned images at publish time (`APP_STORE.md` # Catalog schema). The brain combines the two — image size + `estimated_size` — into the **on-disk footprint** it shows before install (store card + consent dialog; `BRAIN_UI_PROTOCOL.md` # GET /api/v1/catalog/:id/install-plan, `DASHBOARD.md` # Install authorization). It stays advisory: warn on a tight disk, never block. **How to measure it (authoring).** Don't estimate — measure. The import smoke-test already boots the app on a clean install and waits for the health probe; at that point `du -sb` the instance's `data/` volumes and record the result. Apps with a live-boot test can assert the figure against drift on version bumps. Full per-app CI measurement isn't required (some apps need managed services to boot); the author-time measurement during import is the source. @@ -524,7 +524,7 @@ permissions: - **Compose file is verbatim.** Malmo doesn't rewrite it. - **`resources.recommended` is advice, never a cap.** No `limit` field exists in the manifest; authors can't see the user's hardware. Default runtime is uncapped burst; user-set memory caps and control-plane OOM protection live in `APP_ISOLATION.md` # Resource limits. - **Permissions are declared and enforced.** Not just metadata. -- **User content vs. app state are separate stores.** User content (`/home//Photos/`, etc.) accessed by manifest-declared bind mounts of use-case folders; app state in `/var/lib/malmo/instances//data/`. Apps reach user content by reference, never by copy. +- **User content vs. app state are separate stores.** User content (`/home//Photos/`, etc.) accessed by manifest-declared bind mounts of use-case folders; app state in `/var/lib/malmo/state/instances//data/`. Apps reach user content by reference, never by copy. - **`scope: pick-subfolder`** for `folders` — install-time prompt for apps that should manage a subset (notes apps, media subsets). Default is provided by the manifest; user can override. - **Folder source (personal vs household-shared) is installer-elected, not a manifest field.** The manifest declares only the folder + `mode` + `scope`; whether it binds the owner's `~//` or the household `/srv/malmo/shared//` is the installer's per-folder choice (personal instances pick, defaulting to personal; household instances are always shared). Replaces the old `user_folders` / `shared_folders` keys. See `DECISIONS.md` 2026-05-30. - **`folders` mount at a fixed path + injected env var (store apps).** A store manifest declares folder + `mode` + `scope` but no in-container path; the brain mounts each at `/malmo/` and injects `MALMO_FOLDER_`. The app's compose maps that variable to its own library path. `mode` defaults to `read`. Same injection pattern as `MALMO_SERVICE_*` / `MALMO_DATA_DIR`. **Door-2 custom apps diverge:** their verbatim compose has no author to map the env var, so a Door-2 folder grant carries an explicit `target` (the destination path the admin types) and the brain binds straight there. `target` is Door-2-only; store grants omit it (# Custom container — synthetic manifest, `DECISIONS.md` 2026-06-02). diff --git a/docs/specs/BOOT.md b/docs/specs/BOOT.md index 69ac58b..336fe6b 100644 --- a/docs/specs/BOOT.md +++ b/docs/specs/BOOT.md @@ -166,7 +166,9 @@ Upstream Debian ships `smbd.service`, `avahi-daemon.service`, etc. malmo extends ### The bootstrap marker is written by the brain, not by systemd -`malmo-prepare-wizard.service` runs as a oneshot when `/var/lib/malmo-state/.bootstrapped` does not exist. Its job is "ensure the brain is reachable and the wizard can run" — *not* "the box is bootstrapped." The `.bootstrapped` marker is written **by the brain** at the end of the wizard, not by systemd at oneshot exit. This avoids the failure mode where the user closes the wizard halfway and the next boot believes setup is complete. +`malmo-prepare-wizard.service` runs as a oneshot while the box is not yet bootstrapped. Its job is "ensure the brain is reachable and the wizard can run" — *not* "the box is bootstrapped." The marker is written **by the brain** at the end of the wizard, not by systemd at oneshot exit. This avoids the failure mode where the user closes the wizard halfway and the next boot believes setup is complete. + +> **As built, the marker is not a file.** The brain stores `first_run_complete` in its `box_meta` table (`internal/store`). `GET /api/v1/auth/state` reads it and `POST /api/v1/system/first-run-complete` sets it. The `/var/lib/malmo-state/.bootstrapped` file described above was never created, and nor was that folder. The brain's state lives at `/var/lib/malmo/state/` (`STORAGE.md` # mount layout). `malmo-prepare-wizard.service` does not exist yet either — `dist/systemd/` ships the storage and recovery units plus `host-agent.service`. So if that unit is built later, it must read the brain's state, not a file next to it. ### Hang protection on the storage chain diff --git a/docs/specs/BRAIN_HOST_PROTOCOL.md b/docs/specs/BRAIN_HOST_PROTOCOL.md index dcf6f37..6489b73 100644 --- a/docs/specs/BRAIN_HOST_PROTOCOL.md +++ b/docs/specs/BRAIN_HOST_PROTOCOL.md @@ -186,6 +186,17 @@ GET /v1/system/gpu The real host-agent detects the GPU by scanning `/dev/dri/renderD*` and reading each node's PCI vendor (`/sys/class/drm/renderD*/device/vendor`; `0x8086` → `intel`), and resolves the render GID via `os/user.LookupGroup("render")` — the same way `/v1/identity/well-known` resolves `malmo-shared`. The `render` group and the udev rules binding the DRI nodes to it are provisioned by the box build's media stack, not by host-agent (the OS-image half is tracked separately — see issue #125). The fake host-agent returns a synthetic Intel iGPU (`present: true, vendor: "intel"`, a fixed dev `render_gid`) so the override path is exercisable under `make dev`, with a toggle to report `present: false` so the capacity-refusal path is testable without real hardware. This is the brain's only GPU host query — vendor→runtime selection is entirely the brain's job; the manifest stays vendor-agnostic `gpu: true` (`APP_MANIFEST.md` # E). +**System time zone (`POST /v1/system/set-timezone`).** The only write in the "misc host state" group. It is the host op behind the first-run wizard's time-zone step (`FIRST_RUN.md`) and behind Settings → System → Time (`SETTINGS.md`). Pattern A, no response body: + +``` +POST /v1/system/set-timezone { "zone": "Europe/Stockholm" } + → 200 OK +``` + +- `zone` is an IANA tz database name, like `"Europe/Stockholm"` or `"UTC"`. **The brain checks it before calling.** host-agent does not check it. +- The real host-agent runs `timedatectl set-timezone `. That is what re-points `/etc/localtime` (`TIME.md` # System TZ). Both build profiles have it, so a hosted box sets its time zone the same way an appliance does. +- Clock **sync** is a different thing, and it is not a write. The `clock-not-synced` detector reads `chronyc tracking` and reports it through `GET /v1/health/system` (# Health). Nothing in this protocol sets the time. + **Network endpoints (NetworkManager-backed).** host-agent exposes Pattern A routes that wrap NetworkManager's DBus surface: ``` diff --git a/docs/specs/BRAIN_UI_PROTOCOL.md b/docs/specs/BRAIN_UI_PROTOCOL.md index 0c99cfe..aa2e14f 100644 --- a/docs/specs/BRAIN_UI_PROTOCOL.md +++ b/docs/specs/BRAIN_UI_PROTOCOL.md @@ -48,7 +48,7 @@ Plain HTTP. Errors: HTTP status + `{ "code": "...", "message": "...", "details": `GET /api/v1/catalog` returns the browse grid — one `Entry` per app with just what a card needs (`APP_STORE.md` # Catalog schema): id, name, version, `short_description`, `categories`, `icon_url`, the optional `icon_glyph` fallback, and the coarse `footprint`. `GET /api/v1/catalog/:id` returns the detail view: the same `Entry` fields embedded, plus `long_description` (markdown), `screenshot_urls`, `author`, `license`, `links`, and `changelog_url`. Both require an authenticated session (401 if absent); unknown id → 404; a malformed catalog entry → 500 (same integrity posture as install-plan). -`icon_url` and the `screenshot_urls` entries point at `GET /api/v1/catalog/:id/icon` and `/screenshots/:n` — they serve **raw image bytes**, not JSON, so the store loads them directly in `` tags (and they stay out of the OpenAPI surface). `icon_url` is present only when the manifest declares an icon, so the store renders a glyph fallback without ever requesting a 404 — the manifest's optional `icon_glyph` (a Lucide name, returned as a plain JSON string on the `Entry`) picks that fallback glyph; absent ⇒ a generic glyph. The brain resolves these paths inside the app's catalog directory and rejects anything that would escape it; a missing file, an out-of-range index, or a non-numeric `:n` → 404. +`icon_url` and the `screenshot_urls` entries point at `GET /api/v1/catalog/:id/icon` and `/screenshots/:n` — they serve **raw image bytes**, not JSON, so the store loads them directly in `` tags (and they stay out of the OpenAPI surface). `icon_url` is present only when the manifest declares an icon, so the store renders a glyph fallback without ever requesting a 404 — the manifest's optional `icon_glyph` (a Lucide name, returned as a plain JSON string on the `Entry`) picks that fallback glyph; absent ⇒ a generic glyph. **As built (#420) the brain proxies these; it does not read them off disk.** No catalog is baked into the box, and the box keeps none (`APP_STORE.md`). On the first request for an asset the brain fetches it from the control plane and caches the bytes under `MALMO_CATALOG_CACHE_DIR`. That cache is per app and expires after 24 hours. It is not a catalog directory. So what the endpoint serves is the last good copy of a remote file, and a box that has never synced serves nothing. These all return 404: an unknown app, an app with no icon, an index that is out of range, and a non-numeric `:n`. #### GET /api/v1/catalog/:id/install-plan diff --git a/docs/specs/BUILD.md b/docs/specs/BUILD.md index c46cb8a..e7597a8 100644 --- a/docs/specs/BUILD.md +++ b/docs/specs/BUILD.md @@ -270,7 +270,7 @@ All artifacts of a release share the **one** `vX.Y.Z` from the repo `VERSION` fi - `registry.malmo.network/malmo/brain:vX.Y.Z` — the brain image. `latest` tag advances on stable channel. - `registry.malmo.network/malmo/ui:vX.Y.Z` — the dashboard image. Same `vX.Y.Z` as the brain (one repo version); both bundled in the ISO for offline first-boot. - **The control-plane images are published publicly**, and `registry.malmo.network` is a name we can point wherever later (the first realization is `ghcr.io/malmoos/…`, which costs nothing and has no egress bill for public packages). Public rather than private+credential because there is nothing to protect: the brain and UI are built from this public repo, and every secret a box holds is per-box and seeded at provision time (`ENVIRONMENT.md` # Provisioning), never baked into an image. A private registry would buy no confidentiality and would put a pull credential on every box — one more thing to seed, rotate, and fail at 03:00 on a machine nobody can SSH into. Boxes pull **by digest**, not by tag, using the same pinning the app installer already uses (`APP_LIFECYCLE.md`), so a public registry does not mean a mutable one. -- **As-built:** `CI / Cloud image` (`.github/workflows/ci-cloud-image.yml`) additionally attaches `malmo-vX.Y.Z-amd64.raw.xz` + `malmo-vX.Y.Z-amd64.raw.xz.sha256` to the tagged GitHub Release, gated on the same `SHOULD_PUBLISH` condition that used to gate the provider-snapshot upload. That Release asset is the only published **disk-image** artifact: #352 removed the provider-snapshot upload, so the lane holds no hosting-provider credential and a release publishes to no hosting provider. **As of #370 the same lane also pushes the two control-plane images to ghcr** (`ghcr.io/malmoos/brain` and `ghcr.io/malmoos/ui`, tagged `vX.Y.Z` + `latest`), gated on the same `SHOULD_PUBLISH` condition and running *after* the seeded-boot proof — so the images published are the exact local images baked into the disk image that just booted, not a rebuild of them. The push uses the job's own `GITHUB_TOKEN` (`packages: write` — granted at the `cloud-image` job in `release.yml` too, since a called reusable workflow can only narrow its caller's permissions, never widen them). The lane still holds no long-lived registry credential. Pushed digests are written to the job summary. The step runs **after** the Release-asset attach, so a registry failure cannot leave a release without its disk image. **Not yet true in practice:** ghcr creates a package private on first push, and flipping the two packages to public needs a one-time manual change in the package settings by someone with org admin — the workflow cannot do it. Until that flip happens, the images exist but **no box can pull them anonymously**, so the "published publicly" decision above is recorded and not yet in effect. `.raw.xz` names the actual shipped format — this lane's mkosi build produces `.raw` directly (xz-compressed for the upload), not a qcow2 conversion. +- **As-built:** `CI / Cloud image` (`.github/workflows/ci-cloud-image.yml`) additionally attaches `malmo-vX.Y.Z-amd64.raw.xz` + `malmo-vX.Y.Z-amd64.raw.xz.sha256` to the tagged GitHub Release, gated on the same `SHOULD_PUBLISH` condition that used to gate the provider-snapshot upload. That Release asset is the only published **disk-image** artifact: #352 removed the provider-snapshot upload, so the lane holds no hosting-provider credential and a release publishes to no hosting provider. **As of #370 the same lane also pushes the two control-plane images to ghcr** (`ghcr.io/malmoos/brain` and `ghcr.io/malmoos/ui`, tagged `vX.Y.Z` + `latest`), gated on the same `SHOULD_PUBLISH` condition and running *after* the seeded-boot proof — so the images published are the exact local images baked into the disk image that just booted, not a rebuild of them. The push uses the job's own `GITHUB_TOKEN` (`packages: write` — granted at the `cloud-image` job in `release.yml` too, since a called reusable workflow can only narrow its caller's permissions, never widen them). The lane still holds no long-lived registry credential. Pushed digests are written to the job summary. The step runs **after** the Release-asset attach, so a registry failure cannot leave a release without its disk image. **The packages are public now.** ghcr makes a package private on first push. Turning the two packages public needed a one-time change in the package settings by an org admin, which the workflow cannot do. That change is done. `ghcr.io/malmoos/brain` and `ghcr.io/malmoos/ui` both answer an **anonymous** pull, and each carries `latest` plus every released tag from `v0.6.0` on. So the "published publicly" decision above is now real: a box with no registry login can pull the pair its update target names (`UPDATES.md` # 8.4). `.raw.xz` names the actual shipped format — this lane's mkosi build produces `.raw` directly (xz-compressed for the upload), not a qcow2 conversion. ### Channels diff --git a/docs/specs/DECISIONS.md b/docs/specs/DECISIONS.md index 040bde6..ab4a24b 100644 --- a/docs/specs/DECISIONS.md +++ b/docs/specs/DECISIONS.md @@ -21,6 +21,24 @@ Keep entries skimmable. The detailed rationale lives in the affected doc; this f --- +## 2026-08-26 — The brain's state lives under `state/`, and the specs follow the code + +**Previously:** `STORAGE.md` # mount layout put the brain's database at `/var/lib/malmo/brain/state.db`. It put app instances at `/var/lib/malmo/instances//`, and managed-service data under `/var/lib/malmo/managed-services/`. `THREAT_MODEL.md`, `USERS_AND_GROUPS.md` and `LOCAL_ANALYTICS.md` copied those paths. `LOCAL_ANALYTICS.md` had a third version of its own, `/var/lib/malmo-state/brain.db`. `TELEMETRY.md` and `BOOT.md` wrote into a `/var/lib/malmo-state/` root. + +**Now:** the paths in the code stand, and the docs move to match them. The brain writes everything under one folder: `/var/lib/malmo/state/malmo.db` with its `-wal` and `-shm` files, `state/instances//`, and `state/services/-/`. There is no `managed-services/` folder and no `/var/lib/malmo-state/` root. Neither was ever created. **No code change and no migration.** + +**Why:** `/var/lib/malmo/` holds files from two owners. `control-plane/` and `seed.json` are host-agent's. Everything under `state/` is the brain's. So `state/` is the line between them. + +Drop that folder and we have to do one of two bad things. Either point `MALMO_STATE_DIR` at `/var/lib/malmo`, which puts host-agent's files inside the brain's folder. Or teach the brain about two roots. + +One folder also keeps the rest cheap. The dev loop is just `.dev/state/`. The brain container takes one mount. And `internal/hostagent/cpupdate` copies one folder to snapshot the brain around an update. + +The flat spelling in the specs reads a little better. That is the only point on the other side, and it is not worth a migration on boxes that already run. The specs were the stale side, so the docs follow the code (`CLAUDE.md` # Documentation discipline). + +One cost was not cosmetic. `USERS_AND_GROUPS.md` gives a recovery step for a corrupt database, and it named a path the brain never reads. Anyone following it would restore nothing and see no error. That step now names the right path, and says to bring the `-wal` file too. The brain uses WAL mode, so restoring the main file alone gives a working database that is missing its newest writes. + +**Affected docs:** `STORAGE.md` (# mount layout, rewritten to the real tree plus the two silent failures), `THREAT_MODEL.md` (asset table), `USERS_AND_GROUPS.md` (recovery step, plus WAL), `LOCAL_ANALYTICS.md`, `TELEMETRY.md`, `BOOT.md` (note: the first-run marker is `box_meta.first_run_complete`, not a `.bootstrapped` file), `APP_LIFECYCLE.md`, `APP_MANIFEST.md`, `APP_ISOLATION.md`, `FILES.md`, `UPDATES.md`, `NEXT.md`, `CLAUDE.md`. Older entries in this log keep the paths they were written with. + ## 2026-08-17 — The box keeps no catalog on disk; only icons and screenshots are cached, for 24 hours **Previously:** the box was a thin client **with a last-good on-disk cache** (`DECISIONS.md` 2026-07-02). It wrote every verified snapshot to `/var/lib/malmo/catalog-cache/catalog.json` and read it back at boot, so a box that had synced once browsed its last-good catalog forever, online or not. Proxied icons and screenshots were cached next to it with no expiry at all. diff --git a/docs/specs/FILES.md b/docs/specs/FILES.md index 72ddd05..73ceb50 100644 --- a/docs/specs/FILES.md +++ b/docs/specs/FILES.md @@ -23,7 +23,7 @@ Within each root the user browses freely. **The use-case folders are not hardcod **Out of scope, by construction:** -- **App state** (`/var/lib/malmo/instances//`, managed-service data). This is malmo's bookkeeping, "never user-facing" (`STORAGE.md`). The file manager is for user *content*, not the plumbing apps write underneath it. +- **App state** (`/var/lib/malmo/state/instances//`, managed-service data). This is malmo's bookkeeping, "never user-facing" (`STORAGE.md`). The file manager is for user *content*, not the plumbing apps write underneath it. - **Other users' homes.** See # Authorization. - **The rest of the host filesystem** (`/etc`, `/var`, another `/home/`). The two roots are the entire navigable surface. This is a UX boundary; the security boundary is the UID drop in # Execution. - **`~/Shared`** as a *separate* entry. `/home//Shared` is a symlink to `/srv/malmo/shared/` (`STORAGE.md`); the file manager presents the shared tree once, as the **Shared** root, and does not also descend into it through the symlink (avoids a confusing double-listing). diff --git a/docs/specs/LOCAL_ANALYTICS.md b/docs/specs/LOCAL_ANALYTICS.md index e1705a8..ac7eb12 100644 --- a/docs/specs/LOCAL_ANALYTICS.md +++ b/docs/specs/LOCAL_ANALYTICS.md @@ -36,7 +36,7 @@ The Mac mental model: admin can see system logs and who logged in; can't peek at ## Data stores -Two tables in the brain's SQLite (`/var/lib/malmo-state/brain.db`). No second database, no Prometheus, no InfluxDB. +Two tables in the brain's SQLite (`/var/lib/malmo/state/malmo.db`). No second database, no Prometheus, no InfluxDB. ### `events` table @@ -60,7 +60,7 @@ Time-series rollups. Shape: `(timestamp, kind, subject, value)`. | Metric kind | Source | Cadence (full-res) | |---|---|---| -| `app_storage_bytes` | `du` of `/var/lib/malmo/instances//` | Hourly | +| `app_storage_bytes` | `du` of `/var/lib/malmo/state/instances//` | Hourly | | `user_storage_bytes` | `du` of `/home//` | Hourly | | `drive_used_bytes` | `statvfs` per mounted drive (OS drive, data drive) | 10 min | | `app_cpu_percent` | Docker stats API, per container | 60 s | @@ -158,7 +158,7 @@ Out of scope: storage growth attribution ("Photos grew 50 GB this month, mostly - `WEB_UI.md` — top-bar dropdown, per-app page widgets, box overview composition. - `BRAIN_UI_PROTOCOL.md` — `/api/v1/system/live` SSE channel (no-replay, counts against the per-session stream cap); standard REST for historical queries. - `BRAIN_HOST_PROTOCOL.md` — `GET /v1/system/resources`, the host-agent raw-counter sample the brain polls and diffs for the live stream. -- `STORAGE.md` — `/var/lib/malmo-state/brain.db` location, `/var/lib/malmo/instances//` for app storage, `/home//` for user storage. +- `STORAGE.md` — `/var/lib/malmo/state/malmo.db` location, `/var/lib/malmo/state/instances//` for app storage, `/home//` for user storage. ## Open diff --git a/docs/specs/LOGGING.md b/docs/specs/LOGGING.md index 7ce36be..fff2d6a 100644 --- a/docs/specs/LOGGING.md +++ b/docs/specs/LOGGING.md @@ -127,7 +127,7 @@ Caddy on a busy household box can emit thousands of access-log entries per hour, ### Where -A single table `audit_events` in the existing `brain.db`. No new database file, no new connection pool. +A single table `audit_events` in the brain's existing database (`state/malmo.db`). No new database file, no new connection pool. ### Schema sketch @@ -202,7 +202,7 @@ A buggy migration or a future contributor's `DELETE` can't accidentally rewrite Brain doesn't see SSH or SMB logins directly — they go through `sshd` / `smbd` → PAM → journald, outside brain's code path. But "did someone unauthorized access my box?" is exactly the question audit log answers, so these events have to land in `audit_events`. -Brain opens a long-lived `journal_follow` against host-agent filtered to `_COMM=sshd OR _COMM=smbd OR _COMM=sudo OR _COMM=su`. host-agent streams matching entries; brain parses them through a small `pamparse` package and writes `audit_events` rows via the same `audit.Record()` path. The journald cursor checkpoints in `brain.db` (`brain_meta.audit_journal_cursor`) so brain restarts resume cleanly. +Brain opens a long-lived `journal_follow` against host-agent filtered to `_COMM=sshd OR _COMM=smbd OR _COMM=sudo OR _COMM=su`. host-agent streams matching entries; brain parses them through a small `pamparse` package and writes `audit_events` rows via the same `audit.Record()` path. The journald cursor checkpoints in the brain's SQLite (the key-value table is `box_meta`, not a `brain_meta` — `internal/store`) so brain restarts resume cleanly. Event vocabulary added by this ingestion path: diff --git a/docs/specs/NEXT.md b/docs/specs/NEXT.md index 52c00b2..ad0a424 100644 --- a/docs/specs/NEXT.md +++ b/docs/specs/NEXT.md @@ -25,10 +25,18 @@ Each entry: one-sentence shape, the doc it touches, and *why this tier*. The doc `UPDATES.md` # 8 locks the hosted update trigger: the box polls the cloud control plane for its per-box target version, outbound-only. **What authenticates that call is undesigned.** The `enrollment` block the box receives in `seed.json` (`ENVIRONMENT.md` # Admin bootstrap — as built) is an acme-dns account — scoped to writing one DNS TXT record, not to a general control-plane API. So the hosted update path has a designed trigger and no credential to make the call with. +**Where this stands as built (#408).** A hosted box now says which box it is when it asks for its update target: `GET ?box_id=` (`UPDATES.md` # 8.1). That endpoint is public and needs no login, so the identity is a claim, not proof. Anyone who learns a box-id can read what that box is told to run, and can ask as that box. + +We accept that for now, for two reasons. The ask is only a **read** of version information. And the operator path that sets a target sits behind the control plane's own login. + +It stops being enough as soon as the box **writes**. Two things need that: the report-back in `UPDATES.md` # 8.4 step 5 ("this box now runs v0.7.0", or "it failed and rolled back"), and the fleet auto-halt in # 8.5. Both need the control plane to trust who is talking. A fleet view built on an unauthenticated report is one that anyone can lie to. **So build nothing that changes state on top of a bare `box_id`.** + Shape to decide: does the box get a second, separate credential at provision time (simple, one more secret to seed and rotate), or does the `box_id` + a control-plane-issued token become a general-purpose box identity that later features (metering, suspend/restore, fleet management — `ENVIRONMENT.md` # Deferred) also use? The second is more work now and is almost certainly what we end up needing, which is the argument for not designing the first one twice. Also open: rotation, and what a box does when its credential is rejected (keep running and keep serving, is the obvious answer — an auth failure must never take a tenant's apps down). +The **seed is how it gets there.** It is the only per-box channel a real box has, and it is written once (`ENVIRONMENT.md` # Provisioning & first-boot). So the shape to design is a long-lived per-box secret, plus a way to rotate it on the box. + **Context:** `UPDATES.md` # 8.1, `ENVIRONMENT.md` # Updates (hosted) + # Provisioning, `malmoos/cloud` (the other half lives there). -**Why Tier 1:** it blocks implementing the hosted update trigger, which is the first slice of the update work. The apply/rollback half can be built without it; the "what should I be running" half cannot. +**Why Tier 1:** it now blocks the **write** half, not the read half. The trigger itself shipped in #402/#408 on a bare `box_id`, and apply/rollback shipped before it. What cannot be built until this is designed is the box reporting its outcome back (`UPDATES.md` # 8.4 step 5) and the fleet auto-halt that reads those reports (# 8.5). Those are the point of the hosted update path, so the work stops here. --- @@ -364,6 +372,22 @@ Surfaced by `ENVIRONMENT.md`, smaller than the commercial layer but real: (1) ** --- +### Encrypt hosted enrollment credentials at rest (box-side) + +The brain reads per-box acme-dns credentials from the seed and stores them as plain text in `box_meta` (`store.BoxMetaEnrollment`). That matches what the cloud side does today. The risk is someone getting the brain's SQLite file: a leaked subdomain, username and password lets an attacker renew certs for that one box. It does not get them anything more. Encrypt these at rest before the box's database lives on shared or backed-up infrastructure. The cloud side tracks the matching item for its own `boxes` table (`malmoos/cloud` NEXT.md). + +**Context:** `ENVIRONMENT.md` # Provisioning & first-boot, `THREAT_MODEL.md`. +**Why Tier 3:** the damage is limited to one box's certs, and the fix is a box-side change on its own. The open question is where the box keeps its own key, since it has no secret store yet. Settle that before the database moves anywhere shared. + +### Per-app disk quota for hosted tenants + +`ENVIRONMENT.md` # Per-instance resource limits names a per-app **disk quota** as the third dimension the hosted control plane needs to bound a paying tenant. The memory + CPU cgroup limits landed with #211, but disk quota was deferred: the locked storage stack (`ext4` + Docker's `overlay2`, `STORAGE.md`) cannot enforce a per-container write-layer quota portably. Docker's `--storage-opt size=` works only on `xfs`-with-`pquota` or the `devicemapper`/`btrfs` drivers — none of which the appliance or the cloud image use. + +The likely path is **XFS project quotas** on the data tree, driven through host-agent. That is a privileged op: set or clear a project ID and a hard limit on an app's `/var/lib/malmo/state/instances//` subtree and the use-case folders it binds. It needs three things: a verb in `BRAIN_HOST_PROTOCOL.md`, an implementation in `host-agent-real`, and the same store-backed policy and reconcile seam the cgroup limits already use (`internal/store` `instance_resource_limits` would gain a `disk_bytes` column). In practice this is hosted-only. + +**Context:** `ENVIRONMENT.md` # Per-instance resource limits, `STORAGE.md`, `BRAIN_HOST_PROTOCOL.md`. Deferred from #211; tracked in #221. +**Why Tier 3:** it is out of scope until the cloud image's storage layout is settled, and that choice decides whether XFS project quotas are available at all. Settle the shape together with the layout, not after it. + ## Tier 4 — Smaller open items Loose ends. Each is parked until it bites or a higher-tier topic pulls it in. @@ -417,7 +441,7 @@ Loose ends. Each is parked until it bites or a higher-tier topic pulls it in. - Egress allowlist for `internet: true`. `APP_ISOLATION.md`. - Per-app firewall rules (apps as L4 endpoints). `APP_ISOLATION.md`. - Author-declared default/hint for folder source (e.g. an `allow_shared`-style flag) so a manifest can bias the install-time personal-vs-shared toggle without removing the installer's choice. Resolved-for-now as fully installer-elected (`DECISIONS.md` 2026-05-30); revisit if catalog demand appears. `APP_MANIFEST.md` # `folders`. -- fscrypt coverage for per-user app state under `/var/lib/malmo/instances//`. When per-home fscrypt lands, does it extend to managed-service data (per-user Postgres, etc.)? `APP_ISOLATION.md` # Managed services placement, `STORAGE.md` # Future: per-user encryption. +- fscrypt coverage for per-user app state under `/var/lib/malmo/state/instances//`. When per-home fscrypt lands, does it extend to managed-service data (per-user Postgres, etc.)? `APP_ISOLATION.md` # Managed services placement, `STORAGE.md` # Future: per-user encryption. **Storage & first-run** - UTF-8 filename normalization (NFC vs. NFD) across SMB clients — macOS uses NFD on the wire, Linux stores bytes verbatim; "files-first-class" makes this user-visible. `STORAGE.md`. @@ -474,6 +498,7 @@ Loose ends. Each is parked until it bites or a higher-tier topic pulls it in. - Local dev/test subcommands beyond lint/check/resolve (`malmo install --local`, etc.) — let authors run a manifest on their own box before a catalog PR. `malmo manifest lint` (schema + sibling-compose validation, issue #7), `malmo manifest check` (lint + the compose admission policy in one pass, so authors never hand-eyeball `admission.go`), and `malmo manifest resolve` (registry digest + download/disk size resolution into the object-form `images` map, issue #69) shipped and own the `cmd/malmo` skeleton; this remaining item is the heavier "actually install it locally" surface. `APP_MANIFEST.md`, `APP_STORE.md`. - `malmo catalog scaffold --compose ` — deterministic Phase-2 rewrite of an upstream compose into a Door-1 skeleton (drop `ports:`, named volume → `./data/` bind, flag — never silently strip — forbidden directives, emit a `manifest.yml`/`compose.yml` draft with TODOs). Cuts the mechanical keystrokes out of the agent-authoring loop (`docs/dev/authoring-apps-with-an-agent.md`). Deliberately takes a compose the author already located, NOT a repo URL: gathering a compose (it may live in a linked repo, under `docs/`, or only as a `docker run` line) and the ADAPT-DON'T-FORCE bail decision both stay model-side — a tool that crawls or auto-strips gets them wrong. **Trigger: revisit after ~10 hand-authored apps, and build only against the rewrite patterns actually observed across them** — the catalog is two apps deep today, so the common transforms aren't known yet (no premature abstraction, CLAUDE.md). `APP_MANIFEST.md`, `APP_LIFECYCLE.md`. - Catalog PR template + author-facing docs surface (subset of the Tier-3 "Documentation surface" entry). `APP_STORE.md`. +- **`malmo manifest resolve` — size `disk_bytes` without a Docker daemon.** #120 fixed the containerd-store bug by streaming `docker save` and unpacking layer blobs locally. It works, and it works on any store, but it is slow: a multi-GB image like open-webui takes about a minute just to count bytes. The cleaner path is to read the image manifest straight from the registry — no local pull, no `docker save`. Walk the layer descriptors, fetch each compressed blob, unpack it as it arrives, and add up the bytes. Same number, and no Docker daemon needed. Catalog CI wants that too, since no daemon runs there. The registry-client work was scoped in [`../progress/catalog-image-footprint.md`](../progress/catalog-image-footprint.md). `APP_STORE.md` # Catalog schema. - Manifest changelog discipline — when schema v1 → v2 ships, how authors find out. Revisit once we have a `v2` candidate. `APP_MANIFEST.md`. **Web UI** diff --git a/docs/specs/RELEASE_MANIFEST.md b/docs/specs/RELEASE_MANIFEST.md index 4a0e552..1ff29f1 100644 --- a/docs/specs/RELEASE_MANIFEST.md +++ b/docs/specs/RELEASE_MANIFEST.md @@ -122,6 +122,8 @@ This is the load-bearing protection in v1. It is independent of phased rollout ## Failure modes +- **Pointing a test box somewhere else.** `MALMO_RELEASE_BASE_URL` replaces the `releases.malmo.network` base, so a box under test can read a manifest from a local file server. This is safe because **the base URL is not what we trust — the signature is.** A manifest from any host must still verify against the baked keys. So this setting cannot feed a box an unsigned or wrongly signed manifest. Leave it empty for the real releases host. + - **Box can't reach `releases.malmo.network`:** host-agent keeps the last-known manifest in `/var/lib/malmo/manifest.json` (with its signature). Updates pause until connectivity returns. Consistent with `UPDATES.md`: an offline box stays current at its last-applied version. **As built** (`internal/hostagent/relmanifest`): the manifest and its signature share **one** file at that path, and the reason is a crash. Two files means two renames, and a power cut between them leaves the new manifest beside the old signature — a pair that cannot verify, with the previous good manifest already overwritten. The box would then have no usable cache at all, which is not "the previous valid manifest stays in effect". One file is one rename. The cache is also **re-verified when it is read**, so writing `/var/lib/malmo` is not a way around the signature. - **Signature verification fails:** host-agent logs and ignores the manifest. The previous valid manifest stays in effect. A persistent signature failure surfaces as a dashboard warning after 24 hours (operator should investigate; could indicate a CDN/storage corruption or, very rarely, a compromised publishing path). diff --git a/docs/specs/STORAGE.md b/docs/specs/STORAGE.md index 9a921ae..0c8ffa2 100644 --- a/docs/specs/STORAGE.md +++ b/docs/specs/STORAGE.md @@ -106,7 +106,7 @@ Two kinds of app data, two locations: | Kind | Lives at | Survives uninstall? | User sees it? | |---|---|---|---| | **User content** (photo files, music, notes) | `/home//Photos/`, etc. | Yes, always | Yes, daily | -| **App state** (indexes, caches, DBs) | `/var/lib/malmo/instances//data/` | No (or archived on "keep data") | No, ever | +| **App state** (indexes, caches, DBs) | `/var/lib/malmo/state/instances//data/` | No (or archived on "keep data") | No, ever | ### OS drive @@ -151,11 +151,27 @@ Optional but expected. One or more drives, each ext4 + LUKS + TPM-enrolled indep Photos/ Music/ Movies/ Documents/ /var/lib/malmo/ malmo's bookkeeping (brain SQLite, app instances) - brain/state.db - instances// app working dirs — never user-facing - managed-services/ + state/ the brain's state dir (MALMO_STATE_DIR) + malmo.db brain SQLite (+ the -wal / -shm sidecars) + instances// app working dirs — never user-facing + services/-/ managed-service data (e.g. postgres-18/data) + control-plane/ staged control-plane compose + images.json + seed.json provisioning seed (hosted only) ``` +**These are the real paths.** The brain writes everything under one folder. It gets that folder as `MALMO_STATE_DIR`. On a real box it is `/var/lib/malmo/state`. Under `make dev` it is `.dev/state`. host-agent mounts `/var/lib/malmo` into the brain container at the same path, so the host and the brain see the same paths. + +**Why the extra `state/` folder.** `/var/lib/malmo/` holds files from two owners. `control-plane/` and `seed.json` belong to host-agent. Everything in `state/` belongs to the brain, and nothing else writes there. So `state/` is the line between them. + +Without it we would have to do one of two bad things. Either point `MALMO_STATE_DIR` at `/var/lib/malmo`, which puts host-agent's files inside the brain's folder. Or teach the brain about two folders instead of one. + +One folder also keeps other things simple. The brain container needs one mount. And `internal/hostagent/cpupdate` can copy one folder to snapshot the brain before an update (`DECISIONS.md` 2026-08-26). + +Two mistakes here fail **silently**. Neither shows an error: + +- **Restoring the database.** Put it back at `state/malmo.db`, and bring the `-wal` file too. The brain uses WAL mode. If you copy only the main file, you get a working database that is missing its newest writes. +- **Measuring an app's disk use.** Read `state/instances//`, not `instances//`. The second path does not exist, so `du` returns nothing instead of failing. + The use-case folders (`Photos`, `Music`, `Movies`, `Documents`, `Notes`, `Downloads`) are auto-created at user creation. `~/.config/user-dirs.dirs` is populated so XDG-aware tools resolve them. Users may rename, delete, or add folders — apps resolve canonical paths via XDG so a rename doesn't break them. `Movies` (not `Videos`) — matches macOS, signals "stuff you watch" rather than "videos you record." diff --git a/docs/specs/TELEMETRY.md b/docs/specs/TELEMETRY.md index e3323ff..223b697 100644 --- a/docs/specs/TELEMETRY.md +++ b/docs/specs/TELEMETRY.md @@ -58,7 +58,7 @@ Crash *bundles* (full diagnostic dumps with logs, configs, container state) are ## What identifies a box -**Rotating install ID.** A random 16-byte ID, generated locally, rotated every Monday 00:00 UTC. Persisted in `/var/lib/malmo-state/telemetry.json`. The previous week's ID is *not* retained — there is no client-side bridge from week N to week N+1. +**Rotating install ID.** A random 16-byte ID, generated locally, rotated every Monday 00:00 UTC. Persisted under the brain's state dir (`/var/lib/malmo/state/`, `STORAGE.md` # mount layout) — there is no separate `/var/lib/malmo-state` root. The previous week's ID is *not* retained — there is no client-side bridge from week N to week N+1. Consequences: diff --git a/docs/specs/THREAT_MODEL.md b/docs/specs/THREAT_MODEL.md index 0ba6be1..d8ff515 100644 --- a/docs/specs/THREAT_MODEL.md +++ b/docs/specs/THREAT_MODEL.md @@ -31,8 +31,8 @@ Where the trust model does **not** extend: the network (treated as hostile), app |---|---|---| | User content (Photos, Documents, …) | `/home//` (`STORAGE.md`) | Cross-user read; drive theft; ransomware-via-app | | PAM credentials | `/etc/shadow` (`AUTH.md`) | Credential theft, offline cracking | -| App state + per-user managed-service DBs | `/var/lib/malmo/instances//` (`APP_ISOLATION.md`) | Cross-app / cross-user access | -| Brain SQLite (accounts, sessions, audit log) | `/var/lib/malmo/brain/state.db` | Tamper (esp. audit log), session theft | +| App state + per-user managed-service DBs | `/var/lib/malmo/state/instances//` (`APP_ISOLATION.md`) | Cross-app / cross-user access | +| Brain SQLite (accounts, sessions, audit log) | `/var/lib/malmo/state/malmo.db` (+ `-wal`) | Tamper (esp. audit log), session theft | | LUKS keys / TPM seal | OS drive + TPM (`STORAGE.md`) | Whole-box theft, Secure-Boot subversion | | Box network position | nftables, closed-by-default (`MALMO_NETWORK.md`) | Remote exploitation, lateral movement | | Privacy metadata (who-runs-what, audit trail, cloud DNS queries) | brain SQLite + cloud (`LOGGING.md`, `MALMO_NETWORK.md`) | Disclosure / correlation | diff --git a/docs/specs/UPDATES.md b/docs/specs/UPDATES.md index e1846fb..579f8be 100644 --- a/docs/specs/UPDATES.md +++ b/docs/specs/UPDATES.md @@ -207,7 +207,7 @@ The single biggest gap in image-only rollback is **app-managed schema migrations Until lifecycle hooks return (`APP_MANIFEST.md` # F, `APP_LIFECYCLE.md` # Deferred: lifecycle hooks), the brain takes a brute-force snapshot before every app update: -1. **Tar the manifest's declared `data_volumes`** to `/var/lib/malmo/instances//snapshots/pre-update-.tar`. `cache_volumes` are excluded — that's literally what the data/cache split is for (`APP_MANIFEST.md` # C). +1. **Tar the manifest's declared `data_volumes`** to `/var/lib/malmo/state/instances//snapshots/pre-update-.tar`. `cache_volumes` are excluded — that's literally what the data/cache split is for (`APP_MANIFEST.md` # C). 2. **If the app uses a managed service**, `pg_dump` (or equivalent for the service type) the app's logical database into the same snapshot dir. Cheap, well-bounded, runs in the 03:00 window when nothing else is going on. Applies whether or not the service version moved — protects against app-driven schema changes inside the same major. 3. **Retain alongside the previous image for 7 days**, then GC. @@ -425,6 +425,7 @@ The ledger is not bookkeeping. `host-agent` leaves an existing brain container a **As built (#401): steps 1 and 2 are one seam, shared with appliance.** `internal/hostagent/updatetarget` defines a single `Source` ("what should this box be running?") with an implementation per profile — hosted reads a configurable update-target URL over the box's existing outbound path, appliance reads the signed release manifest — and **one** loop consumes it: compare, validate, hold for the window, apply. Neither profile has its own copy of the poll, the compare or the apply. The details that matter: - **The answer names pinned image references, never tags.** `…@sha256:<64 hex>` for both the brain and the UI, resolved once by the sender (see `RELEASE_MANIFEST.md` for the appliance publisher and the cloud's `GET /api/updates/target` for hosted). The box **refuses** an answer that is unpinned, that names only one of the two images, that points at an unexpected repository, or whose carried digest disagrees with its own reference. A refusal is logged and changes nothing — the box stays on its current version, and nothing is pulled. +- **Which repositories count as expected is configuration, not a fixed value.** The default pair is `ghcr.io/malmoos/brain` and `ghcr.io/malmoos/ui` (`BUILD.md` # 6 — public, pulled by digest). Each can be changed on its own with `MALMO_UPDATE_BRAIN_REPO` and `MALMO_UPDATE_UI_REPO`. They exist for two reasons: the CI boot proof serves both images from a registry **inside the guest**, and a box under test may point at another registry. The repository is not what pins the bytes — the digest does that. It is the check that stops a well-formed answer from sending a box to someone else's image. Changing it on a production box is as deliberate an act as pointing that box at a non-default target. - **An unreachable source and a "no target" answer are both no-ops**, and are distinct in the logs. A box never degrades, refuses to serve, or rolls back because it could not ask. - **The window is the hosted apply gate**: 03:00–04:00 local by default (see the window bullet below for where it can come from), which is why the poll is every 15 minutes rather than hourly — an hourly poll can step over an hour-wide window. Within one window a box makes **one** attempt per target version: a failed update has already reverted the box, and retrying it every 15 minutes would be a loop, so the next attempt is the next night. - **The apply goes through host-agent's job lock**, the same one `POST /v1/jobs/system-update` takes, so a target-driven update and an admin-triggered one can never run at once. diff --git a/docs/specs/USERS_AND_GROUPS.md b/docs/specs/USERS_AND_GROUPS.md index 26ec744..bbadcf5 100644 --- a/docs/specs/USERS_AND_GROUPS.md +++ b/docs/specs/USERS_AND_GROUPS.md @@ -80,7 +80,7 @@ If the dashboard is reachable, use it. If it isn't, an admin can: - `docker logs malmo-brain --tail 200` — brain logs. - `systemctl restart malmo-host-agent` — restart the daemon. - `docker restart malmo-brain` — restart the brain. -4. If the brain's SQLite is corrupt: stop the brain, replace `/var/lib/malmo/brain/state.db` from the most recent snapshot (snapshot/restore story is its own doc — not yet written), restart. +4. If the brain's SQLite is corrupt: stop the brain, restore `/var/lib/malmo/state/malmo.db` **and its `-wal` file** from the newest snapshot, then start it again. Restore both. The brain uses WAL mode, so the newest writes may still sit in the `-wal` file. Copy only the main file and you get a working but *older* database, with no error to warn you (`STORAGE.md` # mount layout). `internal/hostagent/cpupdate` copies both files the same way around a control-plane update. The full snapshot and restore story is its own doc, not yet written. **Factory reset short of reinstall** is deliberately not documented here. We don't have a clean "reset everything except user content" story yet; treat reinstall + restore from backup as the floor. diff --git a/web-ui/src/views/FilesView.vue b/web-ui/src/views/FilesView.vue index c2e02d1..1df8192 100644 --- a/web-ui/src/views/FilesView.vue +++ b/web-ui/src/views/FilesView.vue @@ -1,8 +1,9 @@