Repository navigation
Conversation
WaylandYang
left a comment
There was a problem hiding this comment.
Thanks for this. The mechanism is sound and the evidence is unusually thorough. I checked out the branch and ran it here:
- macOS 26.7.1 / Python 3.9.6: all 12 tests in
tests/test_process_lifetime.pypass, including the orphan-then-exec case. - The struct layout matches Apple's
proc_uniqidentifierinfoin xnu-11417.101.15 (p_orig_ppidversionas the fifth field, reserved before 15.4), so the capability gate is right. - The existing harness check (
compileall+run.py --version) passes on the branch, the sandbox wrapperexecs Python so the SIGSTOP handshake works undersandbox-exec, and nothing ondevsince your base touchesisolate.py/run.py.
Requesting changes for four things, the first being the one that blocks a merge:
-
It breaks grading on the production runner.
run_deliveredraisesProcessCleanupErroranywhere that is not macOS, and the oncall grader only catchesTimeoutExpired, so a case that opts in aborts the whole grade inside the Docker runner (Linux). That is where FDE Gym actually runs cases, and #21 just made a container per run the direction. Please fall back tosubprocess.runwhen the host is not macOS (or when delivered code already runs as another user / in a container, where the container's teardown ends orphans anyway), and keep fail-closed only for macOS hosts older than 15.4. If you would rather keep the callback strictly macOS-only, then it needs a name that says so and the case patch must not be the thing that selects it. -
Keep the PR to the harness change. The three
docs/evaluation/*.jsonreceipts,tools/replay_process_lifetime.pyand thepatches/directory are your experiment record, not repository code, and they open three new top-level directories. CONTRIBUTING asks for one change per pull request. Please drop them here; the PR description and the release links already preserve the evidence, and a short note indocs/process-lifetime.mdcan point at them. The one-line case change belongs in a PR to the examples dataset, not in this repository. -
The new workflow lists your own branch under
push.branches; please remove it. Keeping the macOS jobs in a separate workflow is fine. -
Module-level import in
isolate.py.from process_lifetime import run_deliveredworks only because every importer hasharness/onsys.path. Sincerun.pyis the only place that needs it, import it there (whereisolateitself is imported lazily) and leaveisolate.pyas it is.
Also: the tests/ directory is new to the repository and the existing harness CI job does not run it. That is fine for now, but say in CONTRIBUTING.md how to run it, next to the compileall line.
|
|
||
| class _Processes: | ||
| def __init__(self): | ||
| if sys.platform != "darwin": |
There was a problem hiding this comment.
This is the blocking one. On the Docker runner this raises for any case that opted in, and the grader does not catch it, so the grade aborts. Please fall back to subprocess.run off macOS (and in the container / other-UID path) instead of failing closed; keep the hard failure for macOS < 15.4 only.
| "facts": asked_facts(run_dir) if run_dir else set(), | ||
| "run_dir": run_dir, "services": {}} | ||
| "run_dir": run_dir, "services": {}, | ||
| "run_delivered": isolate.run_delivered} |
There was a problem hiding this comment.
If the callback stays macOS-only, this key should say so (run_delivered_macos?), or the fallback should live here so that ctx["run_delivered"] is always safe for a grader to call.
| import sys | ||
| import tempfile | ||
|
|
||
| from process_lifetime import run_delivered |
There was a problem hiding this comment.
Please move this import into run.py next to the lazy import isolate there. isolate is imported from bench.py and npc_server.py too and should not grow a dependency it does not use.
| - tests/test_process_lifetime.py | ||
| - .github/workflows/process-lifetime.yml | ||
| push: | ||
| branches: [main, dev, fix/delivered-process-lifetime-dev] |
There was a problem hiding this comment.
Drop fix/delivered-process-lifetime-dev from the push branches; main and dev plus pull_request is enough.
| @@ -0,0 +1,94 @@ | |||
| """Bounded experiment driver. Official harness/case files remain byte-identical.""" | |||
There was a problem hiding this comment.
The experiment driver, the docs/evaluation/*.json receipts and patches/ should come out of this PR (see the review summary). The release assets you linked already keep them reproducible.
A detached child of an
oncall_incidentresponder can survive its parent's return and access the simulation during later incidents. This adds an opt-inctx["run_delivered"]callback that cleans up identifiable descendants before returning or raising. It requires macOS 15.4+ with validated original-parent identity support; unsupported hosts stop before delivered code runs. Existing graders and the container/UID launch path remain unchanged.Cases are distributed separately. The included one-line patch enables the callback in the pinned public oncall grader; adding the harness alone does not enable cleanup. Do not apply this macOS case patch to Linux runners as-is.
Validation
Fresh paired replay of commit
ea68c1d1a37a3898bf47079d125d4e5b7e144be0againstdevat03c1344d4aba7ba26b6f988fda014ce04f05bb82, using public dataset10fee70cdb7e4010ea9b301359550d930b0de267:All 184 per-case detail entries match in each comparison. Submitted inputs match by hash; both probe processes spawned; errors, timeouts, incidents, model calls and spend were zero. Harness files match the exact commits and remained unchanged during replay. Request evidence is an observer snapshot, not a complete service trace. Fresh source/result hashes and verification receipt.
/usr/bin/python3resolved the earlier local SQLite-view test environment failure.docs/process-lifetime.md, anddocs/evaluation/retain commands, source pins and earlier experimental snapshots.Scope
This is bounded lifecycle cleanup, not a complete OS security boundary. An intermediate process that exits before observation can hide descendants; identity-to-signal races and forcible supervisor termination remain limitations. Apple exposes the identity structure in a private interface, so runtime capability and parent-generation checks are required.
The site smoke reported environment scrubbing without a file sandbox; it is lifecycle integration evidence, not full file/OS isolation. Production object storage, other cases/OSes and upstream CI approval remain separate checks. No score gain or research novelty is claimed.