ABI: Land the ABI 43 batch on current main - #1264
Merged
Merged
Conversation
brandonpayton
added
the
batched-changes
Member
Author
|
Fresh final-tree local validation for the Formula-authority cutover is green. |
Contributor
Phase B-1 matrix build status —
|
| Package | Arch | Status | Sha |
|---|---|---|---|
| icu | wasm32 | failed | — |
| libcurl | wasm32 | failed | — |
| libcxx | wasm32 | failed | — |
| libcxx | wasm64 | failed | — |
| libiconv | wasm32 | failed | — |
| libpng | wasm32 | failed | — |
| libxml2 | wasm32 | failed | — |
| libzip | wasm32 | failed | — |
| openssl | wasm32 | failed | — |
| openssl | wasm64 | failed | — |
| sdl2 | wasm32 | failed | — |
| sdl3 | wasm32 | failed | — |
| sqlite | wasm32 | failed | — |
| sqlite | wasm64 | failed | — |
| zlib | wasm32 | failed | — |
| zlib | wasm64 | failed | — |
| bc | wasm32 | failed | — |
| bzip2 | wasm32 | failed | — |
| coreutils | wasm32 | failed | — |
| cpython | wasm32 | failed | — |
| curl | wasm32 | failed | — |
| dash | wasm32 | failed | — |
| diffutils | wasm32 | failed | — |
| dinit | wasm32 | failed | — |
| erlang | wasm32 | failed | — |
| fbdoom | wasm32 | failed | — |
| file | wasm32 | failed | — |
| findutils | wasm32 | failed | — |
| gawk | wasm32 | failed | — |
| git | wasm32 | failed | — |
| grep | wasm32 | failed | — |
| gzip | wasm32 | failed | — |
| homebrew-bootstrap | wasm32 | failed | — |
| kandelo-sdk | wasm32 | failed | — |
| kernel | wasm32 | failed | — |
| less | wasm32 | failed | — |
| lsof | wasm32 | failed | — |
| m4 | wasm32 | failed | — |
| make | wasm32 | failed | — |
| mariadb | wasm32 | failed | — |
| mariadb | wasm64 | failed | — |
| modeset | wasm32 | failed | — |
| msmtpd | wasm32 | failed | — |
| nano | wasm32 | failed | — |
| ncurses | wasm32 | failed | — |
| netcat | wasm32 | failed | — |
| nginx | wasm32 | failed | — |
| php | wasm32 | failed | — |
| posix-utils-lite | wasm32 | failed | — |
| ruby | wasm32 | failed | — |
| sdl-dsp-test | wasm32 | failed | — |
| sdl2-mixer-playwave | wasm32 | failed | — |
| sed | wasm32 | failed | — |
| spidermonkey | wasm32 | failed | — |
| tar | wasm32 | failed | — |
| tcl | wasm32 | failed | — |
| unzip | wasm32 | failed | — |
| userspace | wasm32 | failed | — |
| vim | wasm32 | failed | — |
| wget | wasm32 | failed | — |
| xz | wasm32 | failed | — |
| zip | wasm32 | failed | — |
| zstd | wasm32 | failed | — |
| bash | wasm32 | failed | — |
| mariadb-test | wasm32 | failed | — |
| mariadb-vfs | wasm32 | failed | — |
| mariadb-vfs | wasm64 | failed | — |
| nethack | wasm32 | failed | — |
| node | wasm32 | failed | — |
| redis-vfs | wasm32 | failed | — |
| spidermonkey-node | wasm32 | failed | — |
| vim-browser-bundle | wasm32 | failed | — |
| nethack-browser-bundle | wasm32 | failed | — |
| rootfs | wasm32 | failed | — |
Auto-generated; replaced on each push. Raw data in the publish-status workflow artifact.
Member
Author
|
Pushed This fixes the first hosted ABI 43 rehearsal failure where Homebrew Focused validation:
The broader publication suite reached its real Homebrew lifecycle but |
Member
Author
|
The ABI 43 verifier recovery is now pushed at 5f27abf. The previous verification path appended a Playwright overlay to the Fresh focused evidence:
The full 17-minute legacy publisher trust suite was intentionally not |
Serialize supported tagged catch state per activation and reject non-reconstructible reference and table state. Require the ABI 43 safety capability across build and host launch boundaries.
Give Node and browser process Workers the same versioned reference graph, external-reference authority, imported state, table replicas, exceptions, and replay-gate owners. Recreate main and side-module Store-local identities before continuation rewind. Release temporary roots on completion, abort, exec, and teardown. Stage dynamic loading as host-only prepare/next operations followed by ordinary Wasm initializers. Serialize pthread loader ownership, replay exact module layouts, and defer signal delivery until libc regains the guest boundary.
Require ABI 43 activation-state-safe capability, exact role metadata, zero native start sections, and staged-loader inventory across package receipts, indexes, sidecars, Homebrew validation, and shell guards. Build valid compiler-EH fixtures normally instead of routing them to an unsupported bucket. Select the declared LLVM archiver on Darwin so final-key source rebuilds remain reproducible. Treat executable kernel.kernel_fork and side-module env.fork as fork-entry evidence in the common binary inventory. Keep the current Homebrew program projection and standalone resolver unchanged; regenerate both once from the coherent batch tip.
Move host-to-kernel structured transfers into bounded, ABI-described scratch regions owned by the kernel. Replace guest-pointer-dependent marshalling for process, IPC, socket, spawn, and signal state with checked wire formats and exact pointer-width layouts. Generate matching Rust, C, and TypeScript metadata, enforce allocation and copy-back bounds, and cover native, wasm32, wasm64, Node, and browser paths. This is the foundational ABI 43 change; later commits close additional ownership and publication seams.
Package cached toolchain sysroots without embedding the source worktree path. Reconstruct the expected layout after cache restore so the action works in a different checkout and on a different runner.
Declare and stage the wasm64 examples needed by browser tests instead of assuming that a previous local build left them in place. Teach CI scope detection, workspace packing, and browser fetch tests to enforce the fixture manifest.
Finish moving blocked retries, VFS operations, IPC, sockets, process state, and scalar results behind kernel-owned scratch and entry gates. Remove remaining host dependence on mutable guest pointer layouts. Apply the same ownership rules in Node and browser workers. Add focused tests for append and offset behavior, environment transactions, worker lifecycle, and fail-stop scratch violations, then refresh the ABI 43 snapshot and generated bindings.
Publish each kernel scratch region and its generated contract as one atomic ABI state. Fail closed when the host observes a missing, stale, or partially initialized region rather than continuing with mixed metadata. Refresh the ABI projection and exercise publication, reusable kernel stacks, and fail-stop validation in the shared Node and browser paths.
Honor the Node bridge endpoint-ownership flags after FIN or cleanup. A close may synchronously reclaim the global pipe slot, so later pumps must neither query nor notify its reusable numeric index.
Give Node conformance runs an explicit, session-owned VFS root and mount lifecycle. Prevent tests from depending on or mutating ambient host storage, and make teardown retire the same mount state that was admitted. Route the libc, POSIX, and Sortix runners through that production host path and cover mount creation, export, and cleanup.
Reject VFS seed entries whose normalized paths shadow an existing session mount or another seed. This keeps the admitted image authoritative and prevents path ordering from silently changing machine state.
Snapshot executable bytes when the resolver admits a program instead of rereading a mutable host file during worker launch. The process now runs the exact image that passed ABI and artifact validation. Cover package resolution and rootfs export while preserving the same behavior in the shared Node and browser launch machinery.
Apply retired-memory debt admission synchronously before ordinary fork creates or copies a child WebAssembly.Memory. Return EAGAIN when the bounded debt limit is saturated, without yielding across the parent snapshot or allocating another complete address space. Keep ordinary fork semantics independent from vfork and prove that the rejected path performs no child allocation or copy.
Add explicit ownership tokens for host workers that intentionally share one process-memory backing. Count the backing once, retire it only after the final alias crosses its quiescence fence, and propagate forced termination until final release. This ownership mechanism does not reinterpret ordinary fork: only callers with an explicit shared-memory transaction may retain an alias.
Drive ordinary-fork admission through the production kernel entry gate instead of constructing a partial worker object. Verify that a retired-memory EAGAIN removes the provisional child and completes the parent mailbox with the truthful error.
Allow a separately launched child worker to borrow the parent process continuation while keeping worker-local replay control state independent. Rebuild main-module and active side-module references from admitted bytes without handing the child authority over parent continuation storage. Gate completion through an explicit lifetime coordinator and cover main, dynamic-link, and active-side replay in Node, Chromium, Firefox, and WebKit fixtures.
Document the fork memory root cause, the shared-memory vfork design, ordinary-fork admission, rejected alternatives, ABI implications, and the evidence still required for release. Add a component resident-set-size harness that separates worker, module, shared-memory, full-clone, and sparse-clone costs. Its results are scoped to those components and do not claim Homebrew application performance.
Binaryen --fpcast-emu rewrites indirectly called functions to a uniform i64 trampoline signature. Host pthread entry calls were still passing a plain JavaScript number, which traps when the wrapper expects BigInt arguments. Build the call arguments from the WebAssembly function parameter count: preserve the plain pointer ABI for ordinary entries and use a BigInt pointer plus zero-filled i64 slots for fpcast trampolines. ABI 43 forward-port: fork-from-thread children now enter through the exported wpk_fork_resume_thread helper. Retain that plain replay ABI and apply fpcast-expanded arguments only where the host calls the table entry directly. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> (cherry picked from commit a2b70da)
Character-special mode is broader than terminal identity. Kandelo encoded host terminal stdio and virtual devices such as /dev/null, framebuffer, audio, and DRM as CharDevice, so terminal probes could mistake any virtual character device for an interactive terminal. Derive terminal identity from the open file description. Dedicated PTY master and slave types are terminals; legacy host stdio is a terminal only when its stable canonical path and host handle agree. Reuse that result for isatty, termios, ioctl namespace gating, and fpathconf. The ABI 43 forward-port retains the current exact 60-byte termios layout. This changes no syscall number, marshalling rule, exported signature, structure layout, generated binding, or VFS ABI metadata. Validation: - native kernel unit suite: 1,491 passed - ABI snapshot and generated bindings check - git diff --check (cherry picked from commit 3b470d3)
Windows does not expose a POSIX permission model through Node. Native entries carry no owner/group/other split or directory search bit, so a guest that drops privileges cannot traverse or write the host-backed sandbox it was given. On Windows only, represent writable host directories as 0777, read-only directories as 0555, writable files as 0666, and read-only files as 0444. Preserve native type bits and let guest chmod and chown metadata override the synthesized permissions. POSIX hosts keep their native modes unchanged. The ABI 43 forward-port performs synthesis only after exact conversion of the native BigInt mode. Existing EOVERFLOW checks for modes, links, sizes, and timestamps remain intact. Validation: - host declaration generation and typechecking - native-metadata and Node uid/gid suites: 15 passed - Windows host execution was not available on this macOS worktree - git diff --check Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> (cherry picked from commit aa12186)
A new git worktree (and a fresh checkout) does not inherit git submodules, a musl sysroot, node_modules, or fetched test binaries, so Vitest and the conformance/browser suites cannot run until those are built or fetched. Agents were treating this as "cannot validate" rather than a setup step. Document the exact sequence in validation.md and build-docs-and-prs.md: submodule init (with the stray-libc/musl recovery), build-musl.sh (sysroot), build.sh (kernel wasm → local-binaries/kernel.wasm + rootfs), root + host `npm ci` (root provides tsx for the conformance runners), and fetch-binaries.sh. Add the explicit instruction not to report "I can't run Vitest/conformance/browser" because a fresh worktree lacks artifacts — build or fetch them and report the real result, or name the exact step that failed. Verified by running the sequence end-to-end in a fresh worktree: kernel wasm + sysroot + rootfs build, `vitest` (778 pass; the only failures were a missing fetched `programs/wasm64/hello64.wasm`), and Sortix conformance (10/10) all ran. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> (cherry picked from commit a3739a2) This conceptual commit combines recovery commits cda80003f and 050a7875e.
Make Area: Purpose the repository convention for PR titles and commit subjects. Add contract-oriented examples and guidance for choosing one primary prefix for cross-cutting changes. Validation: - VitePress documentation build - git diff --check (cherry picked from commit 60fb395)
Chromium-based user agents also contain AppleWebKit. Treating that token as sufficient enabled WebKit-only post-destroy memory handling in Chrome, Edge, Opera, and Chromium on iOS. Forward-port the predicate into the current shared kernel-owned boot helper and keep a pure user-agent classifier for regression coverage. Exclude Chromium, Chrome on iOS, Edge, Opera, Firefox, and Firefox on iOS while retaining Safari and WebKit. Validation: - browser engine detection test: 1 passed - git diff --check (cherry picked from commit 4cbaf2b)
Mutate the sorted mapping vector in place instead of draining and rebuilding it for every munmap call. Preserve the current ABI 43 page-rounding behavior and retain the vector allocation when entries are removed. Keep the original kernel-trap syscall-ring diagnostic while adapting it to the current fatal-entry and blocking-retry cleanup paths. Validated with: scripts/dev-shell.sh -- scripts/ci-run-test-suite.sh cargo-kernel scripts/dev-shell.sh -- npm --prefix host run typecheck End-to-end performance has not been measured yet; that remains part of the batched Node and browser validation. (cherry picked from commit 18a5aa3)
Candidate verification previously appended a Playwright overlay to the selected Formula. That changed the source bytes checked by the exact bottle receipt and stopped every browser-backed Formula test before it could run. Pass the prepared browser directory as an explicit verifier input and project it into the private target verifier home. Keep the native tool home separate and remove the projection during cleanup. Validated with the exact ABI staging verifier fixture, the focused Homebrew Playwright verifier fixture, shell syntax checks, and ABI staging evidence/request-policy freshness checks. The full 17-minute publisher trust suite was intentionally not repeated; its focused verifier fixture was run instead.
Homebrew replaces HOME while running Formula tests, so a verifier-home cache symlink is not observable there. Export the already validated Playwright browser directory explicitly and prove caller poisoning is replaced.
Homebrew rebuilds Formula test environments and drops the ordinary Playwright cache variable. Project the already validated browser directory into the verifier's fresh temporary realm so the existing tap runner can recover it without changing Formula or bottle identity.
Capture the committed secure-exec bit inside the existing kernel entry and carry it into Node and browser replacement workers. This prevents exec and posix_spawn from opening a result-bearing kernel entry while ordered commit work is still draining.
brandonpayton
force-pushed
the
abi43-final-20260813
branch
from
August 17, 2026 05:55
5da3e9e to
c218d35
Compare
Member
Author
|
Refreshing protected-main CI after merging the exact source-test rootfs fix; the ABI 43 head remains unchanged. |
Member
Author
|
Refreshing protected-main workflow authority after the source-test rootfs packing fix; the ABI 43 head remains unchanged. |
Collaborator
|
@brandonpayton Do you confirm we should close the open pull requests that you merged with this one ? For example this pull request I created last month. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Kandelo needs the ABI 43 platform batch to land against the current
mainwithout discarding either the batch's attributed history or the browser CORS
and lazy-shell contracts that landed while the batch was in progress.
PR #1240 no longer merges cleanly. Force-pushing its history would make the
reviewed source harder to recover and would violate its non-destructive merge
expectation. This replacement rebases the same attributed commits onto current
main, then adds purpose-scoped integration fixes.The first hosted bottle rehearsal also exposed a migration blocker: the
temporary Kandelo build-helper bridge still treated legacy package registry
metadata as source authority. That made a tap Formula fail whenever its reviewed
source identity differed from the soon-to-be-retired registry copy, even though
the bottle builder was supposed to build the Formula's source.
Merge method
REBASE MERGE ONLY. DO NOT SQUASH. DO NOT CREATE A MERGE COMMIT.
The 88 non-merge commits preserve intentional authorship and bisection
boundaries. This PR supersedes #1240 once its exact checks and hosted candidate
rehearsal are green.
What changed
main, including PR Restore lightweight lazy shell and browser npm installs #1253.browser worker protocols.
privileged login, sudo, and Ruby product.
/home/userandUser <user@wasm.local>compatibility contract.builds, while retaining fail-closed candidate and publication namespaces.
authority. The temporary bridge now binds only the exact Kandelo build helper;
legacy
package.tomlandbuild.tomlmetadata cannot veto the Formula.Validation
Fresh checks used
scripts/dev-shell.shon the final tree:cargo test -p xtask ... homebrew_tier2_preflight::tests: 23 passedbash scripts/test-homebrew-formula-runtime-closure.shbash scripts/test-homebrew-rootfs-publication-selection.shbash scripts/test-homebrew-patched-launcher.sh: 93 Python tests passed,two intentional skips, plus launcher integration
python3 scripts/test-homebrew-staging-candidate-roots.py: 2 passedruby scripts/check-homebrew-publish-workflow-trust.rbgit diff --checkEarlier validation on the integrated ABI 43 tree also covered:
bash scripts/check-abi-version.shcd host && npm run typecheckbash scripts/test-homebrew-tap-identity.shbash scripts/test-homebrew-staging-candidate-compose.shbash .github/scripts/test-merge-candidate-workflows.shThe first hosted candidate wave reached the real generic bottle builder and
exposed the source-authority mismatch fixed here. Fresh GitHub checks and a new
hosted candidate build/publication wave remain release gates; this PR does not
claim that bottles have already been promoted.
History and attribution
main.this replacement passes CI.