Proxmox VE · VFIO · IOMMU · GPU / PCIe / USB Passthrough

PCI / GPU / USB Passthrough
practical guide for Proxmox VE

Passthrough allows a virtual machine to access selected physical hardware directly through VFIO and IOMMU.
This guide separates the Proxmox host configuration, the VM hardware configuration and the Linux guest driver layer,
so that GPU, PCIe and USB passthrough can be configured without mixing unrelated steps.

GPU / PCIe
USB
IOMMU groups
VFIO binding
GRUB / systemd-boot
Linux guest validation

First separate the layers

The host, the VM and the guest driver are not the same layer

HOST

Proxmox VE node

BIOS IOMMU, kernel command line, VFIO modules, host driver blacklist, initramfs rebuild and binding the selected device to
vfio-pci.

Proxmox VM hardware

q35, OVMF, EFI disk, CPU host, PCI Device, PCI-Express,
ROM-Bar, All Functions and Primary GPU only when the passed GPU is meant to be the VM display output.

GUEST

Linux guest OS

NVIDIA / AMD driver installation, CUDA / ROCm / OpenGL tests, nvidia-smi,
glxinfo and guest-side troubleshooting. The real graphics driver belongs here, not on the Proxmox host.

Core rule: If a GPU is dedicated to passthrough, the Proxmox host should not initialize it with its own graphics driver.
The host should reserve it with vfio-pci; the actual NVIDIA / AMD driver should be installed inside the VM.

00 · Pre-flight inventory

Record the hardware and management path before changing anything

Why this step matters

Passthrough can remove a GPU, NIC, USB controller or storage controller from the Proxmox host.
Before binding any device to vfio-pci, confirm that the host still has a working management path:
web GUI, SSH, IPMI/iDRAC/iLO or a second GPU/iGPU.

Do not pass through the wrong device

Do not pass through the NIC used for Proxmox management unless you have another management interface.
Do not pass through the boot controller used by the Proxmox host.
For a single-GPU host, expect the local console to disappear once the GPU is bound to VFIO.
Prefer a secondary GPU/iGPU/IPMI for the host and dedicate the discrete GPU to the VM.

Show Proxmox and kernel version

pveversion -v
uname -a

List PCI devices with vendor and device IDs

lspci -nn

Find GPU / display / audio functions

lspci -nn | grep -Ei "vga|3d|display|audio|nvidia|amd|ati"

01 · BIOS / UEFI

Enable virtualization and IOMMU features

Intel platform

Intel VT-x
Intel VT-d
Above 4G Decoding, if available
SR-IOV, only if you plan to use SR-IOV capable devices
UEFI boot for modern GPU passthrough

AMD platform

SVM
AMD-Vi / IOMMU
Above 4G Decoding, if available
SR-IOV, only if you plan to use SR-IOV capable devices
UEFI boot for modern GPU passthrough

Resizable BAR note

Above 4G Decoding is often useful for GPU passthrough. Resizable BAR can be beneficial on some systems,
but if the VM has boot or driver initialization problems, test again with Resizable BAR disabled.

Verify IOMMU messages in the Proxmox VE node terminal

dmesg | grep -Ei 'DMAR|IOMMU|AMD-Vi'

Check the kernel parameters actually in use

cat /proc/cmdline

02 · Kernel cmdline

First determine the bootloader: GRUB or systemd-boot

Critical: If Proxmox boots using systemd-boot, editing /etc/default/grub will not change the active boot entry.
If Proxmox boots using GRUB, do not use /etc/kernel/cmdline as the main method.

Detect the bootloader / synced ESP status

proxmox-boot-tool status

Check root filesystem type

findmnt / -o SOURCE,FSTYPE,TARGET

Method A · GRUB

Open the GRUB configuration:

nano /etc/default/grub

Intel compatibility baseline:

GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on iommu=pt"

AMD compatibility baseline:

GRUB_CMDLINE_LINUX_DEFAULT="quiet amd_iommu=on iommu=pt"

Apply the changes and reboot:

update-grub
reboot

Method B · systemd-boot

Open the kernel cmdline file:

nano /etc/kernel/cmdline

The file must remain one single line. Do not delete the existing root=... and boot=... values.
Append only the IOMMU parameters.

Intel example:

root=ZFS=rpool/ROOT/pve-1 boot=zfs quiet intel_iommu=on iommu=pt

AMD example:

root=ZFS=rpool/ROOT/pve-1 boot=zfs quiet amd_iommu=on iommu=pt

Refresh boot entries and reboot:

proxmox-boot-tool refresh
reboot

About modern kernels

On newer Proxmox kernels, IOMMU can already be enabled by default on some platforms. Keeping explicit
intel_iommu=on or amd_iommu=on in the guide is still useful as a clear compatibility baseline,
especially when the article is meant for mixed Proxmox VE versions and different hardware generations.

Do not start with aggressive framebuffer parameters. Use nomodeset, nofb,
video=efifb:off, video=simplefb:off or initcall_blacklist=sysfb_init
only when the Proxmox host still grabs the passthrough GPU after the normal VFIO binding procedure.

03 · Optional framebuffer rescue path

Use this only if the host keeps grabbing the passthrough GPU

Advanced / last resort: These parameters can also affect the host console. They are useful mainly for single-GPU
or primary-GPU passthrough cases where the normal VFIO configuration is not enough.

GRUB rescue example

GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on iommu=pt nofb nomodeset video=efifb:off video=simplefb:off initcall_blacklist=sysfb_init"

update-grub
reboot

systemd-boot rescue example

root=ZFS=rpool/ROOT/pve-1 boot=zfs quiet intel_iommu=on iommu=pt nofb nomodeset video=efifb:off video=simplefb:off initcall_blacklist=sysfb_init

proxmox-boot-tool refresh
reboot

Do not copy this into the VM guest

These framebuffer parameters belong on the Proxmox host only. Inside the Linux VM, normally install the proper GPU driver
and validate the GPU with guest-side tools.

04 · VFIO modules

Load VFIO when the Proxmox host starts

Open /etc/modules

nano /etc/modules

Recommended module list for current Proxmox kernels

vfio
vfio_iommu_type1
vfio_pci

Do not add vfio_virqfd on newer kernels if the module does not exist. It is commonly obsolete on current systems.

Rebuild initramfs for all installed kernels

update-initramfs -u -k all

Verify loaded VFIO modules after reboot

lsmod | grep vfio

05 · Host driver boundary

Do not install unnecessary graphics driver packages on the Proxmox host

Correct boundary: For plain PCIe/GPU passthrough, the Proxmox host usually does not need NVIDIA/AMD drivers,
Xorg libraries, DKMS or build toolchains for the passthrough GPU. Those packages belong inside the guest VM when installing the real GPU driver.

Host side

Enable IOMMU.
Load VFIO.
Bind the selected PCI functions to vfio-pci.
Prevent the host graphics driver from grabbing the passthrough GPU.

Guest side

Install the NVIDIA / AMD driver inside the VM.
Install CUDA / ROCm / OpenGL tools inside the VM if needed.
Validate with nvidia-smi, rocm-smi, glxinfo or workload-specific tools.

06 · Host driver blacklist

Prevent the host from taking control of the passthrough GPU

Use a dedicated blacklist file

Avoid repeated echo ... >> /etc/modprobe.d/blacklist.conf commands. Re-running those commands creates duplicate lines.
A dedicated file is cleaner and easier to remove later.

NVIDIA passthrough GPU: host blacklist file

cat > /etc/modprobe.d/pve-gpu-passthrough-blacklist.conf <<'EOF'
blacklist nouveau
blacklist nvidiafb
blacklist nvidia
blacklist nvidia_drm
blacklist nvidia_modeset
blacklist nvidia_uvm
EOF

AMD passthrough GPU: host blacklist file

cat > /etc/modprobe.d/pve-gpu-passthrough-blacklist.conf <<'EOF'
blacklist radeon
blacklist amdgpu
EOF

Important: Blacklist only the driver family for the GPU that is dedicated to passthrough.
Do not blacklist amdgpu if the Proxmox host uses an AMD iGPU for its own console.

07 · Vendor ID / Device ID

Identify the exact GPU and all related PCI functions

Find GPU / display / audio functions

lspci -nn | grep -Ei "vga|3d|display|audio|nvidia|amd|ati"

Show kernel drivers for the selected PCI slot

lspci -nnk -s 84:00.0
lspci -nnk -s 84:00.1

Example output

84:00.0 0300: NVIDIA Corporation GA104GL [RTX A4000] [10de:24b0] (rev a1)
84:00.1 0403: NVIDIA Corporation GA104 High Definition Audio Controller [10de:228b] (rev a1)

GPU
10de:24b0

GPU audio
10de:228b

In the VFIO configuration, the IDs are written as vendor:device without 0x prefixes.
Always include every function that belongs to the GPU and is required by the VM, usually the VGA/3D/display function and the HDMI/DP audio function.

08 · VFIO binding

Tell Proxmox which PCIe devices must be reserved for the VM

Create the VFIO PCI binding configuration

nano /etc/modprobe.d/vfio-pci.conf

Example binding for GPU + GPU audio

options vfio-pci ids=10de:24b0,10de:228b

Watch the spaces: Do not add spaces after commas in ids=.
Correct is 10de:24b0,10de:228b, not 10de:24b0, 10de:228b.

Optional `disable_vga=1`

If the host still treats the GPU as a VGA device, you can test:

options vfio-pci ids=10de:24b0,10de:228b disable_vga=1

Do not use optional parameters blindly. First test the clean binding line, then add rescue options only if needed.

Rebuild initramfs and reboot

update-initramfs -u -k all
reboot

Verify the specific GPU and audio function after reboot

lspci -nnk -s 84:00.0
lspci -nnk -s 84:00.1

Expected result

Kernel driver in use: vfio-pci

09 · IOMMU groups

Verify PCIe device isolation before assigning the device to a VM

Readable IOMMU group list

for d in /sys/kernel/iommu_groups/*/devices/*; do
  n=${d#*/iommu_groups/*}
  n=${n%%/*}
  printf 'IOMMU Group %s: ' "$n"
  lspci -nns "${d##*/}"
done

Ideal GPU group

IOMMU Group 5: 84:00.0 VGA compatible controller [0300]: NVIDIA Corporation ... [10de:24b0]
IOMMU Group 5: 84:00.1 Audio device [0403]: NVIDIA Corporation ... [10de:228b]

IOMMU group rule: all devices in the same IOMMU group must normally be assigned together to the same VM,
or all left on the host. Do not split one unsafe group across multiple VMs or between host and VM.

ACS override: Use pcie_acs_override=downstream,multifunction only as a last resort.
It can make the group layout look better, but it reduces the isolation guarantee.

10 · VM configuration

Recommended VM settings in Proxmox VE

Machineq35

BIOSOVMF / UEFI

EFI DiskYes

CPU typehost

RAMFixed size preferred

DisplayDefault during install / none for primary GPU

PCI DeviceGPU + audio functions

PCI-ExpressON

All FunctionsPreferred when all functions belong to the GPU

ROM-BarUsually ON

Primary GPUOnly if the passed GPU is the VM display

Manual edit of the VM configuration

nano /etc/pve/qemu-server/100.conf

Secondary / compute GPU example

machine: q35
bios: ovmf
cpu: host
hostpci0: 84:00,pcie=1,rombar=1

Use this style when the GPU is not the primary display output of the VM, or when the VM still uses a virtual display during setup.

Primary GPU display example

machine: q35
bios: ovmf
cpu: host
vga: none
hostpci0: 84:00,pcie=1,x-vga=1,rombar=1

Use this when the passed GPU should be the actual display adapter of the VM.

Combined GPU + audio by slot

hostpci0: 84:00,pcie=1,rombar=1

This is convenient when both functions belong to the same GPU and should move together.

Separated functions

hostpci0: 84:00.0,pcie=1,rombar=1
hostpci1: 84:00.1,pcie=1

Useful when you need explicit control. Keep IOMMU group safety in mind.

11 · Verification inside the Linux VM

Confirm that the guest actually sees and uses the GPU

Check GPU, display and audio functions inside the VM

lspci -nnk | grep -A3 -Ei "vga|3d|display|audio|nvidia|amd|ati"

NVIDIA driver / CUDA visibility check

nvidia-smi

Live NVIDIA GPU monitoring

watch -n 1 nvidia-smi

OpenGL renderer check

glxinfo -B

Simple OpenGL test

glxgears

Guest packages

glxinfo and glxgears are guest-side tools. On Debian/Ubuntu guests they are usually provided by mesa-utils.
They require a graphical session or a configured display stack.

12 · CUDA / ROCm validation

Verify compute acceleration inside the VM

NVIDIA CUDA

nvidia-smi
watch -n 1 nvidia-smi

If PyTorch is already installed inside the VM, you can test CUDA visibility:

python3 - <<'PY'
import torch
print("CUDA available:", torch.cuda.is_available())
if torch.cuda.is_available():
    print("GPU:", torch.cuda.get_device_name(0))
PY

AMD ROCm

rocm-smi

ROCm support depends strongly on GPU generation, Linux distribution, kernel and ROCm version.
Validate the specific AMD GPU against the ROCm compatibility matrix before treating ROCm compute as guaranteed.

13 · USB passthrough

Pass individual USB devices through to the VM

For individual USB devices, you do not need the full PCIe/VFIO GPU procedure. You can pass a USB device directly by Vendor/Product ID.
For high-throughput or latency-sensitive USB workloads, passing through an entire USB controller as PCIe hardware can be more stable.

List USB devices

lsusb

Pass a USB device into the VM by Vendor/Product ID

qm set 804 -usb0 host=058f:6387

Optional USB 3 mode

qm set 804 -usb0 host=058f:6387,usb3=1

Example USB output

Bus 002 Device 013: ID 058f:6387 Alcor Micro Corp. Flash Drive

14 · Troubleshooting

Typical issues and quick checks

No IOMMU detected

Check BIOS settings and active kernel parameters.

cat /proc/cmdline
dmesg | grep -Ei 'DMAR|IOMMU|AMD-Vi'

The host still holds the GPU driver

Verify which driver is currently bound to the card.

lspci -nnk -s 84:00.0
lspci -nnk -s 84:00.1

`vfio_virqfd` is missing

Remove vfio_virqfd from /etc/modules and rebuild initramfs.

nano /etc/modules
update-initramfs -u -k all
reboot

The VM shows a black screen

Check OVMF, q35, vga handling, ROM-Bar and Primary GPU mode.

machine: q35
bios: ovmf
vga: none
hostpci0: 84:00,pcie=1,x-vga=1,rombar=1

GPU works once, then fails after VM reboot

Some GPUs do not reset cleanly. Fully power off the VM, test host reboot, update firmware/BIOS, and check whether the GPU model has known reset issues.

qm stop 100
qm start 100

Wrong device was reserved for VFIO

Remove or correct the wrong ID in /etc/modprobe.d/vfio-pci.conf, rebuild initramfs and reboot.

nano /etc/modprobe.d/vfio-pci.conf
update-initramfs -u -k all
reboot

Final control scheme

Clean separation of responsibility

Proxmox host

BIOS IOMMU → kernel cmdline → VFIO modules → blacklist → vfio-pci binding → verify driver

VM configuration

q35 → OVMF → EFI disk → CPU host → PCIe GPU → ROM-Bar / Primary GPU only as needed

Linux guest

GPU driver → lspci → nvidia-smi / rocm-smi → OpenGL / CUDA / ROCm tests

Most important rule

If the GPU is intended for passthrough, the Proxmox host must not use it through its own graphics driver.
The host should keep it under vfio-pci, and the actual NVIDIA / AMD driver should be installed only inside the VM.

Passthrough gives near bare-metal hardware access, but it reduces flexibility: the device is dedicated to one VM,
live migration is not available for that VM/device combination, and backups/snapshots should be planned with the VM powered off when device state matters.

Technical references

PROXMOX VE