virtio-net: implement RSS (VIRTIO_NET_F_RSS) receive flow steering - #1470
Open
gburd wants to merge 3 commits into
Open
virtio-net: implement RSS (VIRTIO_NET_F_RSS) receive flow steering#1470gburd wants to merge 3 commits into
gburd wants to merge 3 commits into
Conversation
The x86-64 IDT exposes only 224 usable interrupt vectors (32..255), and
every device shares that pool. A guest with many vCPUs and several
multiqueue virtio devices can request one MSI-X vector per queue and
exhaust the pool during boot. Two things then went wrong:
- register_interrupt_handler() called abort() when no IDT vector was
free, crashing the boot instead of letting the device degrade.
- request_vectors() pushed a msix_vector whose allocation had failed,
and ~msix_vector() then unregistered vector 0.
Fix both and add a soft cap so devices stay within budget in the first
place:
- register_interrupt_handler() returns 0 (an impossible real vector,
since 0..31 are CPU exceptions) as an "exhausted" sentinel instead
of aborting. msix_vector's destructor skips the unregister when its
vector is 0, and request_vectors() stops at the first failed
allocation and returns the short vector it did get; the existing
easy_register() already treats a short result as failure, so a
driver falls back to fewer queues rather than crashing.
- Add virtio_driver::reserve_msix_vectors(): a process-wide soft
budget that a multiqueue driver consults to cap how many queues it
arms with a distinct interrupt, leaving headroom for other devices
and always granting at least one. virtio-blk uses it to cap its
active queue count; extra probed virtqueues are simply left unused.
Signed-off-by: Greg Burd <greg@burd.me>
The virtio-net driver hardcoded a single Rx/Tx queue pair drained by one
receive poll thread. Every inbound packet for every connection funnelled
through that one queue and one thread on one CPU, so receive-side network
throughput could not scale past what a single core can drain, regardless of
how many vCPUs the guest had.
Negotiate VIRTIO_NET_F_MQ (and VIRTIO_NET_F_CTRL_VQ, required to activate it)
and drive N Rx/Tx queue pairs:
- Read max_virtqueue_pairs from config space and pick the pair count as
min(advertised pairs, number of vCPUs, MSI-X vector budget).
- Replace the single rxq/txq with vectors of queue pairs. Each Rx queue has
its own poll thread, pinned to a distinct CPU so receivers run in parallel;
each Tx queue has its own xmitter. Transmit selects a Tx queue by CPU id.
- Register one MSI-X vector per data virtqueue (entry = virtqueue index, per
the 1:1 mapping setup_queue() programs) so a completion on queue i wakes
that queue's thread.
- Send VIRTIO_NET_CTRL_MQ_VQ_PAIRS_SET on the control virtqueue to tell the
device how many pairs the guest will use.
When the device does not offer VIRTIO_NET_F_MQ (or offers a single pair, as
with the MMIO transport's shared interrupt) the driver uses exactly one queue
pair and behaves as before.
The per-queue MSI-X vectors are drawn from the shared vector budget added in
the preceding change (virtio_driver::reserve_msix_vectors), so a high-vCPU
guest with this multiqueue net device plus other multiqueue virtio devices
stays within the 224 usable IDT vectors and each device still gets at least
one interrupt-bearing queue instead of exhausting the pool.
Signed-off-by: Greg Burd <greg@burd.me>
Without RSS the host has no hash to distribute inbound flows across the multiqueue Rx virtqueues, so every packet lands on Rx queue 0 and only one receiver thread ever runs even with N queues and N idle CPUs. Negotiate VIRTIO_NET_F_RSS (bit 60) and program the RSS indirection table (round-robin across the N Rx queues), a Toeplitz hash key, and TCP/UDP IPv4+IPv6 hash types via VIRTIO_NET_CTRL_MQ_RSS_CONFIG so the device hashes each flow onto a different Rx queue and N receivers run in parallel. Also fix virtio_driver::get_guest_feature_bit()/setup_features()/ dump_config() to use 1ULL shifts so feature bits >= 32 (RSS is bit 60, VERSION_1 is bit 32) are read correctly. (cherry picked from commit 2a029c9)
Contributor
Author
Follow-up: RSS validated alongside multiqueueWith the downstream dispatch bottleneck fixed (idle-CPU pull, #1472) the multiqueue receive path (#1463) plus RSS flow steering now shows a measured, repeatable gain on a many-connection network workload driven over a real NIC from an external client. RSS hashes inbound flows across the receive queues so that distinct connections land on distinct receiver threads rather than all funneling through one. Under concurrent load the receive work spreads (7 of 8 CPUs busy versus the single-receiver funnel), and mid-to-tail throughput rises ~20-25% across the higher concurrency levels, 0-failed. Without RSS the flows can pile onto one queue even when several exist; with it, N concurrent connections spread across N CPUs. Dependency: this stacks on multiqueue (#1463) which stacks on the MSI-X vector-budget fix (#1465); they should land in that order. Marking ready for review on that basis. |
gburd
marked this pull request as ready for review
August 15, 2026 07:00
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #1463 (multiqueue). Adds RSS receive flow steering so incoming connections are hashed across the multiqueue RX queues implemented in #1463, letting a high-connection-count workload spread receive processing over multiple CPUs.
Measured (stock PostgreSQL, pgbench read-only, on a bare-metal KVM host): with the multiqueue + RSS substrate active, mid-range concurrency throughput improves (c16 ~+34%, c32 ~+16%, c48 ~+12%) versus a single RX queue. This is a generic net-path improvement that benefits any high-connection-count server; PostgreSQL is just the measurement workload.
Draft pending #1463 review (RSS builds on the multiqueue infrastructure there).