Conversation
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
Collaborator
NoteThis is a race condition fix, so 260 clean runs is strong evidence but not mathematical proof of absence elfuse fix-panic (commit 9f98e59) verificationsummary + full per-iteration logs for both the elfuse-native stress test (200 runs) and the mup-level multithreaded-guest test (60 runs). elfuse-fix-panic-verification.log elfuse-teardown-stress.log mup-mt-fork-exec-stress.log |
HVF vCPUs are thread-affine: only the creating thread may run or destroy a vCPU. guest_destroy could call hv_vcpu_destroy on a worker's vCPU from the main thread when the worker was still live past the join cap, freeing the kernel vCPU object out from under its owning thread. The owner's next kernel-side access read the freed, zone-poisoned struct at offset 0x8 and panicked the host (data abort, translation fault, seen twice with an identical backtrace on macOS 26.5.2 / xnu-12377). Close every teardown-vs-live-vCPU window: - thread_destroy_all_vcpus destroys only the main vCPU (owned by the calling thread) and reports any live worker via a live_workers_left out-param instead of destroying it cross-thread. An active non-main slot counts as live -- including a worker still in bring-up (active, vcpu_valid not yet published) about to enter its own vCPU -- unless it has already published vm_exited (a vm-clone child that self-destroyed its vCPU and only stays active for wait4), which holds no live vCPU. - guest_destroy defers all remaining HVF teardown (vcpu destroy, hv_vm_unmap, hv_vm_destroy, munmap of the slab) to process exit when a worker is live: the one operation that safely stops a foreign vCPU. It is always terminal: main() and the fork-child return straight into process exit after it. - Every worker-exit and bring-up-failure path destroys the vCPU under thread_lock and only then clears vcpu_valid / publishes vm_exited, so a teardown scan or a parent wait4 reap can never observe an active-or-reapable slot whose vCPU is still live. - The table-scan interrupt paths (thread_interrupt_all, thread_quiesce_siblings, PTRACE_INTERRUPT) issue hv_vcpus_exit under thread_lock rather than after releasing it, so a handle collected under the lock cannot be handed to HVF after a worker frees it -- the same freed-vCPU fault via hv_vcpus_exit instead of hv_vcpu_destroy. Also resolves the tracked guest_destroy-vs-worker teardown data race (a TSAN-caught unmap racing a live worker): the deferral returns before any teardown write. The residual stall is confirmed to come from uninterruptible host syscalls (blocking fcntl file locks, fsync); the deferral is the correct handling. Close #247
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
HVF vCPUs are thread-affine: only the creating thread may run or destroy a vCPU. guest_destroy could call hv_vcpu_destroy on a worker's vCPU from the main thread when the worker was still live past the join cap, freeing the kernel vCPU object out from under its owning thread. The owner's next kernel-side access read the freed, zone-poisoned struct at offset 0x8 and panicked the host (data abort, translation fault, seen twice with an identical backtrace on macOS 26.5.2 / xnu-12377).
Close every teardown-vs-live-vCPU window:
Also resolves the tracked guest_destroy-vs-worker teardown data race (a TSAN-caught unmap racing a live worker): the deferral returns before any teardown write. The residual stall is confirmed to come from uninterruptible host syscalls (blocking fcntl file locks, fsync); the deferral is the correct handling.
Close #247
Summary by cubic
Fixes host panics from cross-thread
hv_vcpu_destroyby ensuring teardown never destroys a foreign vCPU and defers VM teardown when any worker is live or in bring-up. Also resolves the TSAN unmap‑vs‑worker race inguest_destroy.Bug Fixes
thread_destroy_all_vcpusdestroys only the main vCPU and reportslive_workers_leftwhen any non‑main slot is active (including bring‑up withvcpu_valid=false); slots that already publishedvm_exitedare not counted live.guest_destroydefershv_vm_unmap/hv_vm_destroy/slabmunmap(and any cross‑thread vCPU destroy) to process exit if a worker is live; the main vCPU handle is cleared in all cases.thread_lockbefore clearingvcpu_valid/publishingvm_exited; table‑scan interrupts (thread_interrupt_all,thread_quiesce_siblings,PTRACE_INTERRUPT) now callhv_vcpus_exitwhile holding the lock.cleanup_main_resourcesalways runs host‑only cleanup even when teardown is deferred, so mounts and temp ELF files are not orphaned.test-teardown-live-vcpu-hostandtest-teardown-live-vcpu(spin/bringup), wired intomake checkandtests/test-matrix.sh.Migration
guest_destroyis terminal. If any worker remains live, HV teardown is deferred to process exit; do not reuse the process to start another VM.Written for commit eeec2f4. Summary will update on new commits.