Skip to content

ABI: Restore linear ABI 43 history - #1275

Merged
brandonpayton merged 67 commits into
mainfrom
emdash/abi43-history-replay-final
Aug 17, 2026
Merged

ABI: Restore linear ABI 43 history#1275
brandonpayton merged 67 commits into
mainfrom
emdash/abi43-history-replay-final

Conversation

@brandonpayton

Copy link
Copy Markdown
Member

Why

ABI 43 was prepared as a reviewed linear batch, but PR #1264 was
squash-merged. That preserved the final files while discarding the
conceptual commit history, contributor attribution at the commit level,
and the ancestry needed for future bisects and replay-based staging.

This repair restores a reviewable linear history without changing the
already-shipped ABI 43 tree. It also includes the follow-up spawn
liveness fix needed by the msmtpd bottle path.

What changed

  • Revert the squash commit from PR ABI: Land the ABI 43 batch on current main #1264.
  • Replay all 135 original commits as 63 conceptually complete commits.
  • Keep standalone feature and contributor commits intact.
  • Consolidate contiguous fixup/noise sequences around complete platform,
    POSIX, Homebrew, and continuous integration changes.
  • Record every original commit hash exactly once in replay provenance.
  • Add the reviewed post-commit spawn-liveness repair as one complete
    commit.

History invariants

  • The revert tree exactly equals the squash commit's parent tree.
  • The 63-commit replay tree exactly equals the squash commit tree.
  • All 135 original commit hashes are represented exactly once.
  • The branch is linear and contains 67 commits total.
  • Dependabot and Yannick commits retain their original authorship.
  • Existing co-author attribution is preserved.
  • The net code difference from main is only the spawn-liveness repair;
    the remaining net additions are the replay and repair design/plan
    documents.

Validation

Local structural validation completed:

  • revert and replay tree-object equality
  • exact 135-entry provenance-set equality
  • 67 commits and zero merge commits
  • git diff --check
  • expected seven-file net diff

The spawn-liveness change passed its focused 9-test parity suite and the
combined 85-test spawn/exec/deferred/PID suite before history assembly.
At the owner's direction, the assembled branch will use this PR's CI as
its final test gate.

Merge requirement

This PR must be merged with Rebase and merge. Do not squash it.

brandonpayton and others added 30 commits August 17, 2026 07:14
Serialize supported tagged catch state per activation and reject
non-reconstructible reference and table state.

Require the ABI 43 safety capability across build and host launch
boundaries.

(cherry picked from commit 4efcab8)
Give Node and browser process Workers the same versioned reference
graph, external-reference authority, imported state, table replicas,
exceptions, and replay-gate owners. Recreate main and side-module
Store-local identities before continuation rewind. Release temporary
roots on completion, abort, exec, and teardown.

Stage dynamic loading as host-only prepare/next operations followed by
ordinary Wasm initializers. Serialize pthread loader ownership, replay
exact module layouts, and defer signal delivery until libc regains the
guest boundary.

(cherry picked from commit 95fad1e)
Require ABI 43 activation-state-safe capability, exact role metadata,
zero native start sections, and staged-loader inventory across package
receipts, indexes, sidecars, Homebrew validation, and shell guards.
Build valid compiler-EH fixtures normally instead of routing them to an
unsupported bucket.

Select the declared LLVM archiver on Darwin so final-key source
rebuilds remain reproducible. Treat executable kernel.kernel_fork and
side-module env.fork as fork-entry evidence in the common binary
inventory.

Keep the current Homebrew program projection and standalone resolver
unchanged; regenerate both once from the coherent batch tip.

(cherry picked from commit e0fe315)
Move host-to-kernel structured transfers into bounded,
ABI-described scratch regions owned by the kernel. Replace
guest-pointer-dependent marshalling for process, IPC, socket, spawn,
and signal state with checked wire formats and exact pointer-width
layouts.

Generate matching Rust, C, and TypeScript metadata, enforce allocation
and copy-back bounds, and cover native, wasm32, wasm64, Node, and
browser
paths. This is the foundational ABI 43 change; later commits close
additional ownership and publication seams.

(cherry picked from commit 2a99670)
Package cached toolchain sysroots without embedding the source
worktree path. Reconstruct the expected layout after cache restore so
the action works in a different checkout and on a different runner.

(cherry picked from commit 2950040)
Declare and stage the wasm64 examples needed by browser tests instead
of assuming that a previous local build left them in place. Teach CI
scope detection, workspace packing, and browser fetch tests to enforce
the fixture manifest.

(cherry picked from commit bd2f8a2)
Finish moving blocked retries, VFS operations, IPC, sockets, process
state, and scalar results behind kernel-owned scratch and entry gates.
Remove remaining host dependence on mutable guest pointer layouts.

Apply the same ownership rules in Node and browser workers. Add focused
tests for append and offset behavior, environment transactions, worker
lifecycle, and fail-stop scratch violations, then refresh the ABI 43
snapshot and generated bindings.

(cherry picked from commit 9dac1cf)
Publish each kernel scratch region and its generated contract as one
atomic ABI state. Fail closed when the host observes a missing, stale,
or
partially initialized region rather than continuing with mixed metadata.

Refresh the ABI projection and exercise publication, reusable kernel
stacks, and fail-stop validation in the shared Node and browser paths.

(cherry picked from commit d17028e)
Honor the Node bridge endpoint-ownership flags after FIN or cleanup. A
close may synchronously reclaim the global pipe slot, so later pumps
must neither query nor notify its reusable numeric index.

(cherry picked from commit 2266566)
Give Node conformance runs an explicit, session-owned VFS root and
mount lifecycle. Prevent tests from depending on or mutating ambient
host
storage, and make teardown retire the same mount state that was
admitted.

Route the libc, POSIX, and Sortix runners through that production host
path and cover mount creation, export, and cleanup.

(cherry picked from commit 9713f16)
Reject VFS seed entries whose normalized paths shadow an existing
session mount or another seed. This keeps the admitted image
authoritative
and prevents path ordering from silently changing machine state.

(cherry picked from commit 9e78b6d)
Snapshot executable bytes when the resolver admits a program instead
of rereading a mutable host file during worker launch. The process now
runs the exact image that passed ABI and artifact validation.

Cover package resolution and rootfs export while preserving the same
behavior in the shared Node and browser launch machinery.

(cherry picked from commit 4255846)
Apply retired-memory debt admission synchronously before ordinary
fork creates or copies a child WebAssembly.Memory. Return EAGAIN when
the
bounded debt limit is saturated, without yielding across the parent
snapshot or allocating another complete address space.

Keep ordinary fork semantics independent from vfork and prove that the
rejected path performs no child allocation or copy.

(cherry picked from commit 077c844)
Add explicit ownership tokens for host workers that intentionally
share one process-memory backing. Count the backing once, retire it only
after the final alias crosses its quiescence fence, and propagate forced
termination until final release.

This ownership mechanism does not reinterpret ordinary fork: only
callers with an explicit shared-memory transaction may retain an alias.

(cherry picked from commit 3fbb512)
Drive ordinary-fork admission through the production kernel entry
gate instead of constructing a partial worker object. Verify that a
retired-memory EAGAIN removes the provisional child and completes the
parent mailbox with the truthful error.

(cherry picked from commit 2de2bb8)
Allow a separately launched child worker to borrow the parent process
continuation while keeping worker-local replay control state
independent.
Rebuild main-module and active side-module references from admitted
bytes
without handing the child authority over parent continuation storage.

Gate completion through an explicit lifetime coordinator and cover main,
dynamic-link, and active-side replay in Node, Chromium, Firefox, and
WebKit fixtures.

(cherry picked from commit d18c1e5)
Document the fork memory root cause, the shared-memory vfork design,
ordinary-fork admission, rejected alternatives, ABI implications, and
the evidence still required for release.

Add a component resident-set-size harness that separates worker, module,
shared-memory, full-clone, and sparse-clone costs. Its results are
scoped
to those components and do not claim Homebrew application performance.

(cherry picked from commit d43c1cc)
Binaryen --fpcast-emu rewrites indirectly called functions to a uniform
i64 trampoline signature. Host pthread entry calls were still passing a
plain JavaScript number, which traps when the wrapper expects BigInt
arguments.

Build the call arguments from the WebAssembly function parameter count:
preserve the plain pointer ABI for ordinary entries and use a BigInt
pointer
plus zero-filled i64 slots for fpcast trampolines.

ABI 43 forward-port: fork-from-thread children now enter through the
exported wpk_fork_resume_thread helper. Retain that plain replay ABI and
apply fpcast-expanded arguments only where the host calls the table
entry
directly.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

(cherry picked from commit a2b70da)
(cherry picked from commit ca584ca)
Character-special mode is broader than terminal identity. Kandelo
encoded
host terminal stdio and virtual devices such as /dev/null, framebuffer,
audio, and DRM as CharDevice, so terminal probes could mistake any
virtual
character device for an interactive terminal.

Derive terminal identity from the open file description. Dedicated PTY
master and slave types are terminals; legacy host stdio is a terminal
only
when its stable canonical path and host handle agree. Reuse that result
for
isatty, termios, ioctl namespace gating, and fpathconf.

The ABI 43 forward-port retains the current exact 60-byte termios
layout.
This changes no syscall number, marshalling rule, exported signature,
structure layout, generated binding, or VFS ABI metadata.

Validation:
- native kernel unit suite: 1,491 passed
- ABI snapshot and generated bindings check
- git diff --check

(cherry picked from commit 3b470d3)
(cherry picked from commit 47b1d02)
Windows does not expose a POSIX permission model through Node. Native
entries carry no owner/group/other split or directory search bit, so a
guest
that drops privileges cannot traverse or write the host-backed sandbox
it
was given.

On Windows only, represent writable host directories as 0777, read-only
directories as 0555, writable files as 0666, and read-only files as
0444.
Preserve native type bits and let guest chmod and chown metadata
override the
synthesized permissions. POSIX hosts keep their native modes unchanged.

The ABI 43 forward-port performs synthesis only after exact conversion
of
the native BigInt mode. Existing EOVERFLOW checks for modes, links,
sizes,
and timestamps remain intact.

Validation:
- host declaration generation and typechecking
- native-metadata and Node uid/gid suites: 15 passed
- Windows host execution was not available on this macOS worktree
- git diff --check

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

(cherry picked from commit aa12186)
(cherry picked from commit 3ce6274)
A new git worktree (and a fresh checkout) does not inherit git
submodules, a
musl sysroot, node_modules, or fetched test binaries, so Vitest and the
conformance/browser suites cannot run until those are built or fetched.
Agents
were treating this as "cannot validate" rather than a setup step.

Document the exact sequence in validation.md and build-docs-and-prs.md:
submodule init (with the stray-libc/musl recovery), build-musl.sh
(sysroot),
build.sh (kernel wasm → local-binaries/kernel.wasm + rootfs), root +
host
`npm ci` (root provides tsx for the conformance runners), and
fetch-binaries.sh. Add the explicit instruction not to report "I can't
run
Vitest/conformance/browser" because a fresh worktree lacks artifacts
— build or
fetch them and report the real result, or name the exact step that
failed.

Verified by running the sequence end-to-end in a fresh worktree: kernel
wasm +
sysroot + rootfs build, `vitest` (778 pass; the only failures were a
missing
fetched `programs/wasm64/hello64.wasm`), and Sortix conformance (10/10)
all ran.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
(cherry picked from commit a3739a2)

This conceptual commit combines recovery commits cda80003f and
050a7875e.

(cherry picked from commit e4ffac7)
Make Area: Purpose the repository convention for PR titles and commit
subjects. Add contract-oriented examples and guidance for choosing one
primary prefix for cross-cutting changes.

Validation:
- VitePress documentation build
- git diff --check

(cherry picked from commit 60fb395)
(cherry picked from commit 60df212)
Chromium-based user agents also contain AppleWebKit. Treating that
token as
sufficient enabled WebKit-only post-destroy memory handling in Chrome,
Edge,
Opera, and Chromium on iOS.

Forward-port the predicate into the current shared kernel-owned boot
helper
and keep a pure user-agent classifier for regression coverage. Exclude
Chromium, Chrome on iOS, Edge, Opera, Firefox, and Firefox on iOS while
retaining Safari and WebKit.

Validation:
- browser engine detection test: 1 passed
- git diff --check

(cherry picked from commit 4cbaf2b)
(cherry picked from commit 7846eb3)
Mutate the sorted mapping vector in place instead of draining and
rebuilding it for every munmap call. Preserve the current ABI 43
page-rounding behavior and retain the vector allocation when entries
are removed.

Keep the original kernel-trap syscall-ring diagnostic while adapting it
to the current fatal-entry and blocking-retry cleanup paths.

Validated with:

  scripts/dev-shell.sh -- scripts/ci-run-test-suite.sh cargo-kernel
  scripts/dev-shell.sh -- npm --prefix host run typecheck

End-to-end performance has not been measured yet; that remains part of
the batched Node and browser validation.

(cherry picked from commit 18a5aa3)
(cherry picked from commit 56f0753)
(cherry picked from commit 076362b)
(cherry picked from commit 7a54409)
POSIX loopback is machine-scoped. Kandelo previously searched only the
sender process socket arena, so an unconnected sendto could report
success while dropping the datagram and a connected socket could retain
a false ECONNREFUSED.

Capture an owned route after the process-local send, release the direct
process-table borrows, and then deliver to one accepting foreign
endpoint. This preserves sender metadata, avoids duplicate delivery
across inherited or SO_REUSEADDR bindings, and clears the provisional
connected-socket error only after successful foreign delivery.

This changes no ABI exports, imports, or repr(C) layouts.

Validated with:

  scripts/dev-shell.sh -- scripts/ci-run-test-suite.sh cargo-kernel
  scripts/dev-shell.sh -- bash scripts/check-abi-version.sh
  scripts/dev-shell.sh -- npm run docs:build

(cherry picked from commit c413bd8)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
(cherry picked from commit 36580fb)
Following stat and fstatat calls on /dev/fd/N and the standard-stream
aliases now return the referenced descriptor’s authoritative metadata.
No-follow calls report stable devfs symlink metadata, readlink exposes
the alias target, and O_NOFOLLOW rejects the final symlink.

Keep devfs directory, getdents, and lstat device/inode metadata
coherent. Add a guest regression exercised through NodeKernelHost and
BrowserKernel. The browser test uses the current minimal runner so it
does not activate unrelated shell-package generations.

This fixes GNU coreutils mistaking /dev/null for stdin without changing
the ABI.

(cherry picked from commit 35f0455)
(cherry picked from commit 3004c2b)
Place Kandelo syscall glue and the C runtime startup object before user
link inputs, preserve the caller source, archive, and library order, and
keep the final musl archive last. This prevents an explicit -lc from
resolving syscall definitions before the platform overrides and avoids
scanning dependency libraries before their consumers.

Adapt the original change to the current prepared-linker and
reproducible
path logic. The Kandelo-native driver already uses the same order.

(cherry picked from commit 915a84b)
(cherry picked from commit 9ef2d3e)
Open the replacement iterator before retiring the active stream, and
keep
the old state authoritative if opening or closing the replacement
sequence
fails.

Add kernel failure coverage and shared host-bridge coverage for throwing
close calls and numeric handle reuse. Adapt the host tests to the
current
Rust-lent destination capability instead of bypassing private state.

(cherry picked from commit e96c381)
(cherry picked from commit 052be9c)
This concept commit consolidates its complete contiguous change from the original ABI 43 batch.

Original-commit: 6254851
Original-commit: 25e9475
Original-commit: e61878b
This concept commit consolidates its complete contiguous change from the original ABI 43 batch.

Original-commit: 7b81eeb
Original-commit: c6c8342
Original-commit: cb860f3
Original-commit: 56357e6
Original-commit: 02eaa22
Original-commit: 3b1680f
Original-commit: 8198822
This concept commit consolidates its complete contiguous change from the original ABI 43 batch.

Original-commit: 4b4127b
Original-commit: 042fec9
Original-commit: 455c3fe
Original-commit: 4467498
This concept commit consolidates its complete contiguous change from the original ABI 43 batch.

Original-commit: b81889e
Original-commit: 1e1c95e
Original-commit: b0cdde5
Original-commit: aa8e09e
Original-commit: b4b55a1
This concept commit consolidates its complete contiguous change from the original ABI 43 batch.

Original-commit: b385984
Original-commit: 156ef54
Original-commit: 5b8e504
Original-commit: 90be7d6
Original-commit: f433d14
Original-commit: 63d3728
Original-commit: 1e2b243
Original-commit: 6b04112
Original-commit: 7a80d12
This concept commit consolidates its complete contiguous change from the original ABI 43 batch.

Original-commit: 3411284
Original-commit: 5b6f006
Original-commit: f1d883c
Original-commit: 1289611
Original-commit: a4966e0
Original-commit: fdb8cb0
This concept commit consolidates its complete contiguous change from the original ABI 43 batch.

Original-commit: e330819
Original-commit: 6dc8816
Original-commit: 4dd2ae4
This concept commit consolidates its complete contiguous change from the original ABI 43 batch.

Original-commit: 9b7844f
Original-commit: acca7c2
Original-commit: b8cf706
Original-commit: 8f46de6
This concept commit consolidates its complete contiguous change from the original ABI 43 batch.

Original-commit: 20b3f97
Original-commit: 07f6852
Original-commit: 6ac7091
This concept commit consolidates its complete contiguous change from the original ABI 43 batch.

Original-commit: 7d01b32
Original-commit: ae23d0d
This concept commit consolidates its complete contiguous change from the original ABI 43 batch.

Original-commit: bfd0697
Original-commit: 90638c0
Original-commit: c218d35
(cherry picked from commit 1aa17208e4b750559bf3a76d23705a84bb880b63)
(cherry picked from commit e5d46399c77d90a5994d190d32ddf445459e1892)
@brandonpayton
brandonpayton enabled auto-merge (rebase) August 17, 2026 14:23
Keep the post-commit child retirement fence without reentering the kernel while the detached spawn transaction is still draining.

Source-commit: b7a42cfd2c130d5a46ef6e9c8c16259be7ef4115
Source-commit: 93880b52acbf5619bbab00aaa67107fc83038a14
Source-commit: 108f47fea11ef9e6e4d0833c0db0267d971196d8
@brandonpayton
brandonpayton force-pushed the emdash/abi43-history-replay-final branch from 601a999 to 63e47d3 Compare August 17, 2026 14:26
@brandonpayton brandonpayton added the ready-to-ship Maintainer attests the exact tested head and requests merge preparation. label Aug 17, 2026
@github-actions github-actions Bot removed the ready-to-ship Maintainer attests the exact tested head and requests merge preparation. label Aug 17, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Cannot prepare merge: the exact tested PR head needs either a qualified approving review or a fresh ready-to-ship application by a repository maintainer/admin.

@github-actions

Copy link
Copy Markdown
Contributor

prepare-merge gate failed. Check the workflow run log: https://github.com/Automattic/kandelo/actions/runs/32040652844. Re-apply the ready-to-ship label after fixing.

@brandonpayton brandonpayton added the ready-to-ship Maintainer attests the exact tested head and requests merge preparation. label Aug 17, 2026
@github-actions github-actions Bot removed the ready-to-ship Maintainer attests the exact tested head and requests merge preparation. label Aug 17, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Cannot prepare merge: latest real non-skipped staging-build for HEAD 63e47d3744ffd67ca3f43bfc7bea96608ebbbf08 concluded failure (run https://github.com/Automattic/kandelo/actions/runs/32039022816). Skipped label-event staging runs are ignored and do not count as validation. Re-apply ready-to-ship once staging-build is green, or let a new commit trigger a fresh run.

@github-actions

Copy link
Copy Markdown
Contributor

prepare-merge gate failed. Check the workflow run log: https://github.com/Automattic/kandelo/actions/runs/32044816031. Re-apply the ready-to-ship label after fixing.

@brandonpayton
brandonpayton merged commit f3286e9 into main Aug 17, 2026
236 of 265 checks passed
@brandonpayton
brandonpayton deleted the emdash/abi43-history-replay-final branch August 17, 2026 16:58
@github-actions

Copy link
Copy Markdown
Contributor

Phase B-1 matrix build status — pr-1275-staging-run-32039022816-attempt-5

ABI v43. 0 built, 74 failed, 74 total.

Package Arch Status Sha
icu wasm32 failed
libcurl wasm32 failed
libcxx wasm32 failed
libcxx wasm64 failed
libiconv wasm32 failed
libpng wasm32 failed
libxml2 wasm32 failed
libzip wasm32 failed
openssl wasm32 failed
openssl wasm64 failed
sdl2 wasm32 failed
sdl3 wasm32 failed
sqlite wasm32 failed
sqlite wasm64 failed
zlib wasm32 failed
zlib wasm64 failed
bc wasm32 failed
bzip2 wasm32 failed
coreutils wasm32 failed
cpython wasm32 failed
curl wasm32 failed
dash wasm32 failed
diffutils wasm32 failed
dinit wasm32 failed
erlang wasm32 failed
fbdoom wasm32 failed
file wasm32 failed
findutils wasm32 failed
gawk wasm32 failed
git wasm32 failed
grep wasm32 failed
gzip wasm32 failed
homebrew-bootstrap wasm32 failed
kandelo-sdk wasm32 failed
kernel wasm32 failed
less wasm32 failed
lsof wasm32 failed
m4 wasm32 failed
make wasm32 failed
mariadb wasm32 failed
mariadb wasm64 failed
modeset wasm32 failed
msmtpd wasm32 failed
nano wasm32 failed
ncurses wasm32 failed
netcat wasm32 failed
nginx wasm32 failed
php wasm32 failed
posix-utils-lite wasm32 failed
ruby wasm32 failed
sdl-dsp-test wasm32 failed
sdl2-mixer-playwave wasm32 failed
sed wasm32 failed
spidermonkey wasm32 failed
tar wasm32 failed
tcl wasm32 failed
unzip wasm32 failed
userspace wasm32 failed
vim wasm32 failed
wget wasm32 failed
xz wasm32 failed
zip wasm32 failed
zstd wasm32 failed
bash wasm32 failed
mariadb-test wasm32 failed
mariadb-vfs wasm32 failed
mariadb-vfs wasm64 failed
nethack wasm32 failed
node wasm32 failed
redis-vfs wasm32 failed
spidermonkey-node wasm32 failed
vim-browser-bundle wasm32 failed
nethack-browser-bundle wasm32 failed
rootfs wasm32 failed

Auto-generated; replaced on each push. Raw data in the publish-status workflow artifact.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants