Skip to content

homebrew: build complete VFS images from static Brewfiles - #959

Closed
brandonpayton wants to merge 2 commits into
mainfrom
homebrew/vfs-base-composition
Closed

homebrew: build complete VFS images from static Brewfiles#959
brandonpayton wants to merge 2 commits into
mainfrom
homebrew/vfs-base-composition

Conversation

@brandonpayton

@brandonpayton brandonpayton commented Jul 15, 2026

Copy link
Copy Markdown
Member

Why

Kandelo needs a reproducible way to turn Homebrew formula selections into a
complete virtual filesystem image. The existing builder could pour bottles into an
empty filesystem, but that image lacked the operating-system files from a
platform base. It also accepted formula roots only as command-line flags, so a
checked-in Brewfile could not define the image.

Evaluating a normal Brewfile during an image build would be unsafe and
non-reproducible because Homebrew Bundle files are executable Ruby. The image
builder instead needs a small, explicit selection contract that never executes
the input and that binds the selected tap, roots, base image, and bottle bytes
to the resulting image.

This provides the build-time half of the Homebrew migration path: a tap and a
static Brewfile can produce a complete, auditable VFS image. Full Homebrew
Bundle behavior remains owned by real Homebrew running inside Kandelo.

What changed

  • Add --base-image <image.vfs|image.vfs.zst> so verified bottles can be
    composed onto an ABI-matched platform image. Reject unlabeled, ABI-mismatched,
    and already-composed bases.
  • Preserve the base image's recorded filesystem capacity by default. Rebuild
    existing inodes only when an explicit, aligned --max-bytes requests a new
    capacity.
  • Add --brewfile <Brewfile> with a Ripper-validated static subset: exactly one
    literal lowercase tap and 1 to 128 literal brew entries. The parser accepts
    comments and blank lines, but rejects options, interpolation, conditionals,
    nested Ruby, other Bundle entry types, and multiple taps without evaluating
    the file.
  • Require the Brewfile tap to match sidecar metadata exactly. Normalize bare
    and fully qualified formula names, reject duplicates, and resolve a bounded
    dependency closure in deterministic dependency-first order.
  • Keep repeatable --package <name> for lower-level callers. It is explicitly
    mutually exclusive with --brewfile.
  • Record the Brewfile hash and byte count, ordered normalized roots and their
    digest, tap commit, base digest, and bottle digests. Bounded VFS metadata
    carries summaries; the external report carries the detailed audit record.
  • Fix duplicate guards that incorrectly treated JavaScript Set.add() as a
    boolean. Duplicate roots, packages, dependencies, and link targets now fail
    closed.
  • Document the static subset, provenance model, and the boundary with full
    guest Homebrew Bundle behavior.

Validation

Run from the repository dev shell on exact commit
e539654fe4910658e1c11e0aa4c8c22122dbdbbf:

  • scripts/test-homebrew-brewfile-selection.sh passed, including dynamic Ruby,
    interpolation, duplicate, foreign-tap, size, count, symlink, NUL, and invalid
    UTF-8 rejection cases.
  • 74 focused VFS image, Homebrew planner, and builder Vitest tests passed.
  • scripts/test-homebrew-tap-native-sidecars.sh passed. It covers dependency
    closure, explicit-package compatibility, static Brewfile composition, base
    ABI and capacity rules, bounded provenance, exact tap identity, and a
    third-party tap.
  • Host TypeScript declaration generation and the full ESM, CommonJS, and
    browser bundle build passed.
  • scripts/check-abi-version.sh and git diff --check passed.

The full host Vitest suite was also attempted, but this artifact-light worktree
cannot provide a green broad-suite claim: 90 test files and 1,107 tests passed;
30 files and 53 tests failed, primarily because kernel and program Wasm
fixtures are absent, plus one unrelated artifact-guard timeout. The focused
Homebrew tests passed inside that run.

Scope and remaining evidence

The image-building command is Node.js-only because it reads local metadata,
bottles, Brewfiles, and images. The resulting VFS bytes are host-neutral. A real
Node or browser boot of a multi-package Brewfile image was not run, so the next
acceptance rung still needs to boot this output on both hosts.

The static builder intentionally supports one tap. Multi-tap composition needs
an explicit cross-tap metadata and provenance contract. Full dynamic Brewfile
DSL remains available only through real Homebrew inside a running Kandelo
machine.

This does not change the Kandelo kernel ABI and does not rebuild or publish any
package, bottle, release artifact, or existing VFS image. Newly composed VFS
and report bytes include the new selection provenance.

This is a source PR for the broad non-ABI Homebrew release-control batch. It
should be absorbed unchanged and validated with that batch rather than merged
on its own.

Homebrew-built application images need an existing platform root filesystem, but the VFS builder previously could only pour bottles into an empty image. Accept an explicit ABI-matched platform base while preserving truthful bottle and filesystem provenance.

Restore the base with its recorded growth ceiling, rebase only for an explicitly different capacity, and reject unlabeled, ABI-mismatched, or previously Homebrew-composed inputs. Bind only the base hash, byte count, and ABI into output metadata while retaining full source metadata in the external report, so large attestations cannot overflow output metadata or claim the mutated bytes.

Record canonical tap identity, preserve unchanged base inodes during default composition, and cover first-party and third-party taps, near-limit metadata, composition rejection, retained files, and both capacity paths.
@brandonpayton brandonpayton changed the title homebrew: build bottle VFS images on platform bases homebrew: build complete VFS images from static Brewfiles Jul 15, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Phase B-1 matrix build status — pr-959-staging

ABI v39. 0 built, 6 failed, 6 total.

Package Arch Status Sha
kandelo-sdk wasm32 failed
rootfs wasm32 failed
shell wasm32 failed
lamp wasm32 failed
node-vfs wasm32 failed
wordpress wasm32 failed

Auto-generated; replaced on each push. Raw data in the publish-status workflow artifact.

@brandonpayton

Copy link
Copy Markdown
Member Author

Absorbed by rebase-merged #967. Main is now 6efb411 with the sealed tree b133bfa018f7e2bc84c4845edf2f945b2f844954, and ABI 41 activation completed successfully in run 29550069085. The absorption audit confirmed this PR\x27s behavior and tests are present in that merged tree, so this source PR is closed and its branch is deleted.

@brandonpayton
brandonpayton deleted the homebrew/vfs-base-composition branch July 17, 2026 02:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant