sched: env-tunable idle spin-before-halt (OSV_IDLE_SPIN) - #1476
Draft
gburd wants to merge 1 commit into
Draft
Conversation
… (OSV_IDLE_SPIN) An idle CPU with no runnable thread spins polling incoming_wakeups a fixed 10000 iterations before halting via arch::wait_for_interrupt, which under a hypervisor is a VM-exit. For a request/reply server that runs one blocking worker per connection, each worker is typically re-woken by its next request a short time after it blocks; when the spin is too short to catch that wake, the halt and the subsequent wake-IPI cost a round-trip of several VM-exits per request. At high connection counts this halt/wake round-trip -- not any lock, fault, or TLB cost -- becomes the throughput ceiling: the guest saturates its VM-exit rate while most exits are halts and their wakes. Make the spin-before-halt count tunable via the OSV_IDLE_SPIN environment variable. The default is unchanged (10000), so an unset variable preserves the historical behavior byte-for-byte; a workload that benefits from trading idle CPU cycles for fewer halt/wake round-trips raises it. A longer spin lets a CPU catch a soon-arriving wake in the poll loop and skip the halt entirely, which collapses the per-request halt/wake VM-exit round-trip and lifts throughput at high concurrency, at the cost of burning cycles on a genuinely idle system -- hence opt-in rather than a new default.
gburd
marked this pull request as ready for review
August 18, 2026 09:47
gburd
marked this pull request as draft
August 18, 2026 09:51
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
sched: env-tunable idle spin-before-halt (
OSV_IDLE_SPIN)An idle CPU with no runnable thread spins polling
incoming_wakeupsa fixed10000 iterations before halting via
arch::wait_for_interrupt, which under ahypervisor is a VM-exit. For a request/reply server that runs one blocking
worker per connection, each worker is typically re-woken by its next request a
short time after it blocks. When the spin is too short to catch that wake, the
halt and the subsequent wake-IPI cost a round-trip of several VM-exits per
request. At high connection counts this halt/wake round-trip becomes the
throughput ceiling: the guest saturates its VM-exit rate while most exits are
halts and their wakes, and the CPUs are otherwise idle.
This makes the spin-before-halt count tunable via the
OSV_IDLE_SPINenvironment variable so a busy server can spin slightly longer before halting
an idle CPU and thereby avoid the halt/wake round-trip per request.
Behavior
OSV_IDLE_SPINunset the count is 10000, so theidle path is byte-for-byte the historical behavior. A zero or unparseable
value also falls back to 10000.
halt/wake round-trips raises it (for example
OSV_IDLE_SPIN=100000). A longerspin burns cycles on a genuinely idle system, which is why it is opt-in rather
than a new default.
Effect
Under sustained wakeup load, a longer spin lets a CPU catch a soon-arriving wake
in the poll loop and skip the halt entirely, which collapses the per-request
halt/wake VM-exit round-trip and reduces halt/wake churn. The win grows with
concurrency (more round-trips avoided) and there is no low-concurrency
regression, since a lightly loaded system reaches a runnable thread or the halt
well within the spin either way.
Scope
One file (
core/sched.cc), +24/-2. Generic base scheduler code; does not dependon any other change and is independently reviewable. Applies directly on
master.