vfio_assigned_device: allow fixed BAR addresses#4001
Conversation
There was a problem hiding this comment.
Pull request overview
Enables VFIO-assigned devices to pre-program PCI BARs using either host-reported physical BAR addresses or explicit fixed physical addresses, propagating this configuration through the CLI, ttrpc management schema, and VFIO resource handles. This supports devices (e.g. nvgrace GPU variant drivers) whose coherent-memory BAR physical address is not discoverable via sysfs resource.
Changes:
- Replace per-BAR passthrough booleans (
barN=pt) with a structured per-BAR address policy (GuestAssigned/HostAssigned/Fixed(u64)), and update CLI syntax tobarN=hostorbarN=0x<addr>. - Extend the ttrpc management API schema with
VfioBarAddressentries and parse them into the resource handle config. - Validate and apply configured BAR addresses against discovered BAR layout (including address/size alignment), and update Guide documentation.
Reviewed changes
Copilot reviewed 9 out of 9 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| vm/devices/pci/vfio_assigned_device/src/resolver.rs | Threads the renamed BAR configuration field through resolver paths. |
| vm/devices/pci/vfio_assigned_device/src/lib.rs | Implements BarAddressConfig-driven BAR pre-programming + validation and adds unit tests. |
| vm/devices/pci/vfio_assigned_device_resources/src/lib.rs | Introduces BarAddressConfig and updates VFIO resource handle structs to carry it. |
| openvmm/openvmm_ttrpc_vmservice/src/vmservice.proto | Adds VfioBarAddress and bar_addresses to the management API schema. |
| openvmm/openvmm_entry/src/ttrpc/mod.rs | Parses proto BAR address entries into BarAddressConfig for resource construction. |
| openvmm/openvmm_entry/src/lib.rs | Plumbs CLI-parsed BAR config into VFIO resource handles. |
| openvmm/openvmm_entry/src/cli_args.rs | Updates --vfio parsing to accept barN=host and barN=0x<addr> and adds tests. |
| Guide/src/user_guide/openvmm/vfio.md | Documents updated --vfio BAR pinning syntax and the fixed-address use case. |
| Guide/src/reference/openvmm/management/cli.md | Updates CLI reference example from bar0=pt to bar0=host. |
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 10 out of 10 changed files in this pull request and generated 1 comment.
Comments suppressed due to low confidence (1)
vm/devices/pci/vfio_assigned_device/src/lib.rs:838
apply_bar_addressesindexesbar_masks[i + 1]and later writesbars[i + 1]wheneveris_64bitis true, but the function itself doesn't guardi + 1 < 6. Today the caller rejects a 64-bit BAR at index 5, but keeping the bounds check local avoids a potential panic if this helper is reused or if the pre-validation changes.
let is_64bit = cfg_space::BarEncodingBits::from(bar_flags[i]).type_64_bit();
if !is_64bit && address > u32::MAX as u64 {
anyhow::bail!("BAR {i} is 32-bit but address {address:#x} exceeds 4 GB");
}
let address_mask = if is_64bit {
The nvgrace-gpu VFIO variant driver synthesizes a coherent-memory BAR whose physical address is not reported through the PCI sysfs resource table. OpenVMM therefore cannot discover that BAR's host address, even though it must be placed at the same guest address for peer-to-peer DMA on platforms where transactions bypass IOMMU translation. Allow each VFIO BAR to use either the host-reported address or an explicit fixed address. Carry this policy through the CLI, management service, and VFIO resource boundary, and validate fixed addresses against the discovered BAR layout before programming them. This replaces the opaque `barN=pt` CLI syntax with `barN=host`; the old spelling is no longer accepted. The management schema gains equivalent structured support for host-derived and fixed BAR addresses.
4a530af to
61e40a1
Compare
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 10 out of 10 changed files in this pull request and generated 1 comment.
Comments suppressed due to low confidence (1)
vm/devices/pci/vfio_assigned_device/src/lib.rs:846
apply_bar_addressesindexesbar_masks[i + 1]/ writesbars[i + 1]for 64-bit BARs without first ensuringi + 1 < 6. If the device reports a (malformed) 64-bit BAR at index 5 and the user pins it, this will panic with an out-of-bounds access. Please reject 64-bit BARs that start at BAR5 before usingi + 1.
let is_64bit = cfg_space::BarEncodingBits::from(bar_flags[i]).type_64_bit();
if !is_64bit && address > u32::MAX as u64 {
anyhow::bail!("BAR {i} is 32-bit but address {address:#x} exceeds 4 GB");
}
let address_mask = if is_64bit {
(bar_masks[i + 1] as u64) << 32 | (bar_masks[i] as u64 & !0xf)
} else {
The nvgrace-gpu VFIO variant driver synthesizes a coherent-memory BAR whose physical address is not reported through the PCI sysfs resource table. OpenVMM therefore cannot discover that BAR's host address, even though it must be placed at the same guest address for peer-to-peer DMA on platforms where transactions bypass IOMMU translation.
Allow each VFIO BAR to use either the host-reported address or an explicit fixed address. Carry this policy through the CLI, management service, and VFIO resource boundary, and validate fixed addresses against the discovered BAR layout before programming them.
This replaces the opaque
barN=ptCLI syntax withbarN=host; the old spelling is no longer accepted. The management schema gains equivalent structured support for host-derived and fixed BAR addresses.