generalize crate for multi-device PCIe passthrough#1573
Open
cheese-head wants to merge 3 commits into
Open
Conversation
Signed-off-by: Patrick Riel <priel@nvidia.com>
cheese-head
requested review from
a team,
derekwaynecarr,
maxamillion and
mrunalp
as code owners
Collaborator
|
@elezar adding this for your review as well since it's adjacent to gpu work |
…ng restart reconciliation, without rebinding or mutating sysfs. Signed-off-by: Patrick Riel <priel@nvidia.com>
elezar
reviewed
May 27, 2026
Comment on lines
+96
to
+102
| let vendor = read_sysfs_trimmed(&dev_dir.join("vendor"))?; | ||
| if vendor != NVIDIA_VENDOR_ID { | ||
| return Err(VfioError::NotNvidia { | ||
| bdf: bdf.to_string(), | ||
| vendor, | ||
| }); | ||
| } |
Member
There was a problem hiding this comment.
Question: Is this code NVIDIA-specific? If so, we may want to update the function name.
elezar
reviewed
May 27, 2026
| static VFIO_ID_REFCOUNTS: LazyLock<Mutex<HashMap<String, usize>>> = | ||
| LazyLock::new(|| Mutex::new(HashMap::new())); | ||
|
|
||
| pub(crate) fn current_driver_name(sysfs: &SysfsRoot, bdf: &str) -> Option<String> { |
Member
There was a problem hiding this comment.
We introduced an abstraction over sysfs paths to enable testing. Does an abstraction for a device also make sense? We have a number of places where we:
- Get the path for a specific
bdf - Append a path to it
- Read a link or contents of a file to determine a property.
Would hiding this behind named methods be useful?
Signed-off-by: Patrick Riel <priel@nvidia.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Generalize
openshell-vfiobeyond GPU-only / single-device passthrough so it can serve as the binding/validation primitive layer the VM driver needs to implement RFC-0004'sresource_requirementsmodel. Adds atomic IOMMU-group binding, dry-run validation forValidateSandboxCreate-style paths, class-agnostic device enumeration, and a correctness fix for partially-failed binds. Purely additive plus one bug fix; consumer crates are unchanged.Related Issue
Foundational for RFC-0004 sandbox resource requirements (#1360). Unblocks multi-device-per-sandbox passthrough that the existing single-device API could not express when devices shared an IOMMU group (consumer GPUs + HDA + USB-C, multi-PF NICs, devices behind ACS-deficient PCIe switches).
Changes
prepare_pci_group_for_passthrough/release_pci_group_from_passthroughfor atomic bind/release of multiple PCIe devices sharing one IOMMU group. Rollback only restores devices newly bound by the call, so it does not steal bindings owned by other guards.validate_pci_for_passthrough/validate_pci_group_for_passthroughas dry-run pre-flight checks forValidateSandboxCreate-style paths. Performs every structural and IOMMU-peer check without touchingdriver_overrideor any other kernel state.prepare_*now delegates to its validate counterpart to keep the two in lockstep.probe_host_vfio_candidates(sysfs, vendor_filter)for vendor-filtered, class-agnostic enumeration of passthrough-eligible PCI devices so consumers can advertiseDeviceClassCapabilityfor arbitrary classes (GPUs, NICs, VFs) instead of being limited toprobe_host_nvidia_vfio_readiness.PciBindGuard::companion_bdfs()accessor for consumer-side persistence of grouped bindings (crash-recovery state, status reporting).VfioError::GroupMismatchandVfioError::EmptyGroupfor typed validation responses.bind_device_to_vfioto cleardriver_overrideand re-probe the host driver ondrivers_probefailure and on post-probe polling timeout. Previously a failed bind could leave the device wedged withdriver_override="vfio-pci"pinned on disk, causing the next probe event to silently re-bind to vfio-pci.Testing
mise run pre-commitpassescargo test -p openshell-vfiopasses (52/52, up from 32)cargo clippy -p openshell-vfio --all-targets -- -D warningscleancargo check -p openshell-driver-vmclean (consumer crate compiles unchanged)Checklist
docs/coversopenshell-vfiotoday