Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
67 commits
Select commit Hold shift + click to select a range
108bde5
Revert "ABI: Land the ABI 43 batch on current main"
brandonpayton Aug 17, 2026
c584aed
Fork: Make replay activation state safe
brandonpayton Jul 25, 2026
7788863
Host: Rebuild replay references in fresh workers
brandonpayton Jul 26, 2026
147ee99
Packages: Bind replay artifacts and rollout to ABI 43
brandonpayton Jul 26, 2026
1f48a5b
Host: Bound kernel scratch transfers
brandonpayton Jul 25, 2026
1cc586a
Build: Make cached toolchain sysroots portable
brandonpayton Jul 26, 2026
d3612d5
Build: Materialize browser memory64 fixtures
brandonpayton Jul 26, 2026
d7d5ed0
Host: Complete kernel scratch ownership closure
brandonpayton Jul 27, 2026
09925fe
Host: Publish and validate kernel scratch atomically
brandonpayton Jul 27, 2026
c6b9ddd
Host: Stop pumping relinquished TCP endpoints
brandonpayton Jul 27, 2026
5f89a3c
Host: Run conformance on owned storage
brandonpayton Jul 28, 2026
9b4e583
Host: Reject shadowed session seeds
brandonpayton Jul 28, 2026
59a2d4c
Host: Pin mutable executable bytes
brandonpayton Jul 28, 2026
53b864f
Host: Reject saturated ordinary forks before cloning
brandonpayton Jul 31, 2026
a637474
Host: Retain exact shared memory aliases
brandonpayton Jul 31, 2026
ff4d465
Tests: Exercise fork admission through sealed authority
brandonpayton Aug 1, 2026
decaee8
Fork: Borrow replay workers across main and side modules
brandonpayton Jul 31, 2026
4c4fe9f
Performance: Record affordable-fork design and replay evidence
brandonpayton Jul 31, 2026
7d6389b
Host: Call fpcast-emulated pthread entries correctly
brandonpayton Jul 3, 2026
25abd57
POSIX: Distinguish terminals from character devices
brandonpayton Jul 25, 2026
998959e
Host: Synthesize POSIX permissions for Windows mounts
mho22 Jul 15, 2026
a83225b
Docs: Document fresh-worktree validation prerequisites
brandonpayton Jul 9, 2026
9b8a2ec
Docs: Require purpose-prefixed PRs and commits
brandonpayton Jul 20, 2026
6f3c476
Browser: Restrict reclaim handling to WebKit
brandonpayton Jun 11, 2026
9a091e8
Performance: Avoid munmap mapping-vector churn
brandonpayton Jun 20, 2026
f8110f7
Kernel: Reject late reaped-process syscalls
brandonpayton Jun 22, 2026
9feb7e9
Network: Deliver loopback UDP across processes
brandonpayton Jul 9, 2026
50cec7e
Kernel: Preserve descriptor identity through devfs aliases
brandonpayton Jul 11, 2026
f486278
SDK: Preserve executable linker input order
brandonpayton Jul 12, 2026
03590c8
POSIX: Preserve directory streams when rewind fails
brandonpayton Jul 22, 2026
54d2fec
Host: Reflect ABI 43 modules from admitted bytes
brandonpayton Aug 1, 2026
d85a9a1
Host: Deliver caught signals before retrying waits
brandonpayton Jul 29, 2026
b3b1147
Host: Ignore debug names while patching thread modules
brandonpayton Jul 11, 2026
73c6dee
Host: Finalize readiness timeouts through the kernel
brandonpayton Jul 11, 2026
1815cca
Host: Run Node service demos from shared dinit images
brandonpayton Jun 16, 2026
2cd6f2e
CI: Repair xtask fixtures and gate xtask tests
brandonpayton Jul 2, 2026
b4a3248
CI: Gate the Rust workspace as one contract
brandonpayton Jul 6, 2026
f8c0f88
Host: Reuse one bundled source worker entry
brandonpayton Jun 25, 2026
b9ba14c
Browser: Add image-owned file ingest
brandonpayton Jul 10, 2026
2070304
Build: Update mkrootfs esbuild security release
dependabot[bot] Jul 27, 2026
acd9a7c
Build: Refresh minor and patch npm dependencies
dependabot[bot] Jul 27, 2026
3ed55f7
Build: Adopt Node 26 type definitions
dependabot[bot] Jul 27, 2026
0438288
ABI: Move lifecycle metadata ownership into Rust
brandonpayton Aug 17, 2026
34782e6
Audio: Provide process-safe OSS PCM across hosts and packages
brandonpayton Aug 17, 2026
30dcd12
ABI: Establish the production vfork mechanism
brandonpayton Aug 17, 2026
53cbb2e
VFS: Default executable mounts to nosuid
brandonpayton Aug 11, 2026
69974ee
POSIX: Make credentials and set-ID execution authoritative
brandonpayton Aug 17, 2026
46040dd
POSIX: Complete interrupted waits and login integration
brandonpayton Aug 17, 2026
f1b6c74
Build: Initialize the reentrant Node runtime
brandonpayton Aug 17, 2026
b4e8e0e
Homebrew: Integrate ABI 43 descriptors and products
brandonpayton Aug 17, 2026
c3db517
CI: Repair ABI 43 product contract fixtures
brandonpayton Aug 17, 2026
007d1f0
Homebrew: Make Formula candidate provenance authoritative
brandonpayton Aug 17, 2026
f5580a7
Homebrew: Preserve candidate Formula tests and program projections
brandonpayton Aug 17, 2026
4edeb0c
Homebrew: Preserve candidate bottle identity
brandonpayton Aug 17, 2026
2d982ed
ABI: Unblock first-wave candidate publication
brandonpayton Aug 17, 2026
94b74ab
ABI: Enforce qualified candidate metadata
brandonpayton Aug 17, 2026
3b5c9a8
Homebrew: Pour authenticated candidates in bounded realms
brandonpayton Aug 17, 2026
d0b9023
Homebrew: Preserve Chromium across Formula tests
brandonpayton Aug 17, 2026
cc8e0b7
ABI: Materialize candidate dependency bottles
brandonpayton Aug 17, 2026
ff4eea4
[CI] Keep publisher integration out of PR staging
brandonpayton Aug 16, 2026
e68a0e1
[ABI] Export PHP side-module ABI identity
brandonpayton Aug 16, 2026
16fff15
Host: Preserve worker output process identity
brandonpayton Aug 16, 2026
43dc01a
ABI: Authorize artifact-independent staging validation
brandonpayton Aug 17, 2026
0d31c0f
Host: Carry secure exec and retirement through commit
brandonpayton Aug 17, 2026
48b5761
docs: design the ABI 43 history replay
brandonpayton Aug 17, 2026
414ae5c
docs: plan the ABI 43 history replay
brandonpayton Aug 17, 2026
63e47d3
Host: Defer spawn liveness until after allocation
brandonpayton Aug 17, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
460 changes: 460 additions & 0 deletions docs/superpowers/plans/2026-08-17-abi43-history-replay.md

Large diffs are not rendered by default.

138 changes: 138 additions & 0 deletions docs/superpowers/plans/2026-08-17-posix-spawn-postcommit-liveness.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,138 @@
# POSIX Spawn Post-Commit Liveness Implementation Plan

> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.

**Goal:** Let a committed `posix_spawn` child reach host Worker allocation without re-entering the kernel while its post-commit transaction is still draining.

**Architecture:** Remove the redundant pre-allocation child-liveness query from both host entries. Keep the existing post-allocation query as the single authoritative fence before registration, and lock that ordering into the shared spawn parity test.

**Tech Stack:** TypeScript, Vitest, Node worker host, browser worker host

## Global Constraints

- Preserve the kernel-entry reentrancy guard unchanged.
- Preserve Node/browser observable process behavior.
- Do not add an msmtpd-specific path.
- Do not register a Worker after a child exits during an asynchronous allocation.

---

### Task 1: Enforce post-allocation liveness ordering

**Files:**
- Modify: `host/test/spawn-host-parity.test.ts`
- Modify: `host/src/node-kernel-worker-entry.ts`
- Modify: `host/src/browser-kernel-worker-entry.ts`

**Interfaces:**
- Consumes: `CentralizedKernelWorker.shouldLaunchPendingChild(pid: number): boolean`
- Produces: one post-allocation liveness fence in each `handlePosixSpawn`

- [ ] **Step 1: Write the failing ordering regression**

Extract each `handlePosixSpawn` body with the test's existing helper. Locate
`createFreshProcessMemory`, `shouldLaunchPendingChild`, and `registerProcess`.
Assert that `shouldLaunchPendingChild` occurs exactly once, after allocation
and before registration:

```ts
expect(spawn.match(/shouldLaunchPendingChild/g)).toHaveLength(1);
expect(spawn.indexOf("createFreshProcessMemory")).toBeLessThan(
spawn.indexOf("shouldLaunchPendingChild"),
);
expect(spawn.indexOf("shouldLaunchPendingChild")).toBeLessThan(
spawn.indexOf("registerProcess"),
);
```

- [ ] **Step 2: Run the regression and verify RED**

Run:

```bash
scripts/dev-shell.sh bash -lc \
'cd host && npx --no-install vitest run test/spawn-host-parity.test.ts'
```

Expected: FAIL for both Node and browser because each entry contains two
liveness queries and the first precedes allocation.

- [ ] **Step 3: Implement the minimal shared fix**

Delete only the first `shouldLaunchPendingChild` branch from each
`handlePosixSpawn`. Add a short comment explaining that the successful
prepared-target commit proves the child at callback entry and that the retained
post-allocation check owns kill-during-yield handling.

- [ ] **Step 4: Run focused spawn and lifecycle tests**

Run:

```bash
scripts/dev-shell.sh bash -lc \
'cd host && npx --no-install vitest run \
test/spawn-host-parity.test.ts \
test/exec-state-tracking.test.ts \
test/deferred-worker-start.test.ts \
test/spawn-pid-authority.test.ts'
```

Expected: all selected tests pass.

- [ ] **Step 5: Commit the implementation**

```bash
git add host/test/spawn-host-parity.test.ts \
host/src/node-kernel-worker-entry.ts \
host/src/browser-kernel-worker-entry.ts
git commit -m "fix: defer spawn child liveness until after allocation"
```

### Task 2: Validate and publish the runtime repair

**Files:**
- Verify: all files from Task 1
- Verify: `abi/snapshot.json`

**Interfaces:**
- Consumes: the Task 1 branch
- Produces: a reviewable Kandelo pull request and hosted msmtpd retry input

- [ ] **Step 1: Run the complete host suite**

```bash
scripts/dev-shell.sh bash scripts/ci-run-test-suite.sh vitest
```

Expected: all host tests pass, including every resource-isolated case.

- [ ] **Step 2: Verify the ABI snapshot is unchanged**

```bash
scripts/dev-shell.sh bash scripts/check-abi-version.sh
```

Expected: the ABI check passes without changing `ABI_VERSION` or the snapshot.

- [ ] **Step 3: Review the exact branch diff**

```bash
git diff --check origin/main...HEAD
git status --short
git log --format=fuller origin/main..HEAD
```

Expected: only the approved design/plan, two host entries, and parity test are
changed; known submodule dirt remains unstaged.

- [ ] **Step 4: Open the purpose-first pull request**

Use a title describing the process-lifecycle outcome. Put `## Why` before
`## What changed`, list the exact validation, and state that hosted msmtpd is
not yet proven until its staging retry succeeds.

- [ ] **Step 5: Retry msmtpd after merge**

Dispatch the immutable ABI 43 request against the new Kandelo source only after
the runtime PR merges. Require the original standalone msmtpd service test to
build and pass before calling the end-to-end defect fixed.
156 changes: 156 additions & 0 deletions docs/superpowers/specs/2026-08-17-abi43-history-replay-design.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,156 @@
# ABI 43 History Replay Design

## Purpose

Pull request #1264 was squash-merged as `bd28679cd2452f24e5c3ea2c245ed3dfcace1e05`.
That preserved the final source tree but discarded the reviewable ABI 43 commit
sequence from `c218d35225c411859d067133242316ddf07cb08e`. Repair the public history
without rewriting or force-pushing `main`.

The repair is one pull request merged with GitHub's **Rebase and merge** method.
It first reverts the squash, then replays a conceptually complete ABI 43
sequence, and finally adds the postcommit `posix_spawn` liveness repair needed
by the `msmtpd` bottle test.

## Repository facts

- Squash commit: `bd28679cd2452f24e5c3ea2c245ed3dfcace1e05`
- Squash parent/current pre-ABI tree:
`c1ff05541a7d9eaa4f295acb065f735cdb0bd272`
- Original PR head: `c218d35225c411859d067133242316ddf07cb08e`
- Original PR merge base:
`847875deeb07d2be4039485a7c3287ad5edf400f`
- Original PR range: 135 commits
- GitHub rebase-merge limit: 100 commits

The original PR head is not byte-identical to the squash commit. Two protected
source-test preparation commits landed on `main` after the original branch
point. GitHub's squash result correctly combined those intervening changes
with the ABI 43 diff, affecting three protected workflow-support files. The
replay must therefore be applied onto the squash parent, not copied from the
old PR head as a complete tree.

## One-PR topology

The repair branch starts at the current squash-merged `main` and contains this
linear sequence:

1. One revert of the squash commit.
2. Sixty-three conceptually complete ABI 43 replay commits.
3. The reviewed design and implementation plan.
4. One postcommit spawn-liveness commit containing its contract docs,
Node/browser implementation, and parity regression.

The tree immediately after the revert must equal the squash parent. The tree
immediately after the ABI replay must equal the squash commit exactly. The
final pull-request diff may then contain only the replay documentation and the
spawn-liveness repair.

## Consolidation policy

Keep a commit separate when it represents an independently reviewable platform
contract, has a distinct upstream author, or is an independent dependency
update. Consolidate contiguous commits when later commits complete or repair
the same contract, regenerate artifacts for that contract, or record its
validation and rollout.

Each consolidated commit records every original SHA and subject in its body.
The resulting commit author is the author shared by the grouped commits.
`mho22` remains the author of the Windows mount-permissions commit. Each of the
three Dependabot commits remains separate with Dependabot as author. Any
original co-author trailer is retained on the consolidated commit.

The following table is the complete grouping authority. Ordinals refer to
`git log --reverse 847875dee..c218d3522`.

| Original ordinals | Replay treatment | Concept |
|---|---|---|
| 1-38 | Keep individually | Independent fork, host, POSIX, network, SDK, CI, and browser contracts |
| 39-41 | Keep individually | Three independent Dependabot updates |
| 42-49 | Consolidate | Move lifecycle and host metadata ownership into Rust |
| 50-52 | Consolidate | Provide process-safe OSS audio across hosts and packages |
| 53-57 | Consolidate | Establish the ABI 43 vfork mechanism and build artifacts |
| 58 | Keep individually | Default executable mounts to `nosuid` |
| 59-66 | Consolidate | Make credentials, set-ID execution, and PTY metadata authoritative |
| 67-72 | Consolidate | Complete interrupted-wait, login, sudo-lite, and vfork integration |
| 73-75 | Consolidate | Pin build tools and initialize the reentrant Node runtime |
| 76-83 | Consolidate | Integrate ABI 43 Homebrew descriptors and product projections |
| 84-86 | Consolidate | Repair CI and product contract fixtures for ABI 43 |
| 87-93 | Consolidate | Make Formulae and candidate provenance authoritative |
| 94-97 | Consolidate | Preserve candidate Formula tests and program projections |
| 98-102 | Consolidate | Preserve keg, launcher, rebuild, schema, and public clone identity |
| 103-111 | Consolidate | Unblock and validate first-wave candidate publication |
| 112-117 | Consolidate | Enforce qualified metadata and close first-wave platform drift |
| 118-120 | Consolidate | Pour authenticated candidates in bounded verification realms |
| 121-124 | Consolidate | Preserve and project Chromium across Formula tests |
| 125-127 | Consolidate | Materialize exact candidate dependency bottles |
| 128 | Keep individually | Keep publisher integration out of PR staging |
| 129 | Keep individually | Export PHP side-module ABI identity |
| 130 | Keep individually | Preserve worker output process identity |
| 131-132 | Consolidate | Make exact ABI validation artifact-independent and authorize the builder |
| 133-135 | Consolidate | Carry secure exec and retirement through commit and refresh projections |

This produces 63 replay commits. With the revert, two documentation commits,
and the final runtime repair, the pull request has 67 commits.

## Replay mechanics

Build a temporary replay line from the squash parent. Apply each group in
original order with `git cherry-pick --no-commit`, resolve only the protected
source-test overlap introduced after the original merge base, and commit the
group once its complete final state is present. Never select conflict sides by
blanket `ours` or `theirs`; compare each conflict against the squash tree.

After the grouped line is complete, require:

```text
git diff --exit-code <grouped-replay-head> bd28679cd2452f24e5c3ea2c245ed3dfcace1e05
```

Then, on the public repair branch, revert the squash and cherry-pick the 63
grouped commits. Re-run the same tree-equality check before adding the runtime
repair.

## Merge and attribution rules

- No force push to `main`.
- No squash merge.
- No merge commit.
- The pull request must be merged using GitHub **Rebase and merge**.
- The pull request must contain fewer than 100 commits.
- Every replay commit must have the intended original author.
- Consolidated commit bodies must list their complete original SHA set.
- Compare original and replay author inventories before merge.

GitHub rewrites committer identity and commit SHAs during rebase merge while
retaining authors. That is acceptable; losing original authors or conceptual
boundaries is not.

## Validation

Before opening the pull request:

1. Prove the post-revert tree equals `c1ff05541...`.
2. Prove the post-replay tree equals `bd28679cd...`.
3. Prove the final diff contains only the design/plan and spawn-liveness files.
4. Prove the branch is linear and contains fewer than 100 commits.
5. Compare author inventories and original-SHA coverage.
6. Run `scripts/check-abi-version.sh`.
7. Run the focused spawn/exec host tests.
8. Run the full Vitest lane used by the merged ABI tree.
9. Run `git diff --check`.

Before merging, re-fetch `main`, require the pull-request base to remain the
expected squash commit, and re-run the tree, attribution, and commit-count
checks. If `main` moves, rebuild the replay against the new base rather than
force-merging stale history.

## Staging interaction

The active tap workflow may continue validating the immutable original ABI 43
request while this history repair is prepared. Its candidate bytes remain
useful. The repaired Kandelo history produces a new source commit identity, so
after the history pull request lands the staging coordinator must either prove
reuse against the new request or rebuild only contracts changed by the final
spawn-liveness repair. It must not pretend the old source commit is the new
one.
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
# POSIX Spawn Post-Commit Liveness Design

## Problem

The ABI 43 kernel-entry gate correctly forbids a second kernel WebAssembly
export while a serialized ingress or detached protocol transaction is still
draining. Node's `handlePosixSpawn` violates that boundary immediately after a
successful prepared-target commit: it calls `shouldLaunchPendingChild`, which
enters `kernel_get_process_exit_signal`. The msmtpd standalone service test
reaches this path and fails with `KernelReentrantEntryError` before its child
Worker can be attached.

This is shared host-runtime behavior, not an msmtpd Formula defect. A Formula
workaround would hide a general `posix_spawn` sequencing error.

## Design

Remove the pre-allocation `shouldLaunchPendingChild` call from both Node and
browser `handlePosixSpawn` implementations. The callback is reached only after
Rust has committed the exact pending child's prepared target. Node has not
yielded since that commit, so another ingress cannot have killed the child.
Browser may have yielded while draining unrelated process teardowns, but it
still performs the authoritative liveness check after memory allocation and
before registration, so removing the earlier optimization cannot resurrect a
dead child.

Retain the post-allocation check in both hosts. It runs after an asynchronous
allocation boundary, releases the unused memory lease when the child is no
longer live, and prevents Worker registration for an exited child.

Do not weaken `KernelEntryGate`, expose an entry capability to detached host
callbacks, special-case msmtpd, or delay Node launch merely to make the
forbidden query legal.

## Validation

Extend the existing spawn host-parity contract to require that neither entry
calls `shouldLaunchPendingChild` before `createFreshProcessMemory` completes,
and that both retain exactly one check after allocation and before process
registration. Run that test red before implementation and green afterward.

Then run the focused spawn/lifecycle host tests, the complete host Vitest
suite, and the ABI snapshot check. The hosted msmtpd Formula retry remains the
end-to-end proof because it exercises the exact Node worker, kernel, shell,
network, and `posix_spawn` path that exposed the defect.
7 changes: 4 additions & 3 deletions host/src/browser-kernel-worker-entry.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3056,9 +3056,10 @@ async function handlePosixSpawn(
const secureExec = kernelWorker.takeCommittedExecSecureExec(childPid);
await waitForProcessTeardowns();

// Unrelated teardown waits yield to the event loop. Keep a successfully
// created zombie, but never resurrect it with a new Worker.
if (!kernelWorker.shouldLaunchPendingChild(childPid)) return 0;
// The shared launcher already committed the exact pending child. The
// post-allocation fence below owns any exit observed across this teardown
// wait or allocation yield; do not add a separate kernel entry while the
// postcommit transaction is still draining.
post({ type: "proc_event", kind: "spawn", pid: childPid, ppid: parentPid });

const { programBytes, programModule, argv } = program;
Expand Down
7 changes: 4 additions & 3 deletions host/src/node-kernel-worker-entry.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2710,9 +2710,10 @@ async function handlePosixSpawn(
envp: string[],
): Promise<number> {
const secureExec = kernelWorker.takeCommittedExecSecureExec(childPid);
// Preserve a child that became a zombie before launch, but do not resurrect
// it by registering a new execution generation.
if (!kernelWorker.shouldLaunchPendingChild(childPid)) return 0;
// The shared launcher invokes this callback only after Rust committed the
// exact pending child. Do not re-enter the kernel while that postcommit
// transaction is still draining; the first legal liveness fence follows
// the asynchronous memory allocation below.
post({ type: "proc_event", kind: "spawn", pid: childPid, ppid: parentPid });

const { programBytes, programModule, argv } = program;
Expand Down
23 changes: 23 additions & 0 deletions host/test/spawn-host-parity.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -181,6 +181,29 @@ describe("spawn host parity", () => {
}
});

it("both spawn adapters check child liveness only after allocation yields", () => {
for (const entry of [nodeEntry, browserEntry]) {
const handler = posixSpawnHandlerSource(readFileSync(entry, "utf8"));
const allocation = handler.indexOf("createFreshProcessMemory(");
const liveness = handler.indexOf("shouldLaunchPendingChild(childPid)");
const registration = handler.indexOf("registerProcess(childPid");
expect(
handler.match(/shouldLaunchPendingChild\(childPid\)/g) ?? [],
`${entry} must retain exactly one post-allocation liveness fence`,
).toHaveLength(1);
expect(allocation, `${entry} must allocate process memory`)
.toBeGreaterThanOrEqual(0);
expect(
liveness,
`${entry} must check the child after allocation yields`,
).toBeGreaterThan(allocation);
expect(
registration,
`${entry} must check liveness before registering the Worker generation`,
).toBeGreaterThan(liveness);
}
});

it("both exec adapters consume the complete commit-captured transition", () => {
for (const entry of [nodeEntry, browserEntry]) {
const handler = execHandlerSource(readFileSync(entry, "utf8"));
Expand Down
Loading
Loading