VFIO "GPU Passthrough" Guide for Proxmox

BIOS Settings:

• Disable Above 4G
• Disable ReBAR
• Disable SR-IOV (Leave this enabled if using a Quadro or other SR-IOV enabled graphics accelerator)
• Enable IOMMU Support
• Enable VT-d
• Ensure primary output in BIOS is set to iGPU (NOT AUTO)
  

Proxmox Kernel Version:

• Ensure Proxmox is running on kernel version 6.0.2 or later - 

  pveversion -v

• Update Proxmox -

• apt update
  apt dist-upgrade
  reboot

Proxmox Config:

nano /etc/kernel/cmdline
root=ZFS=rpool/ROOT/pve-1 boot=zfs
Change to:
For Intel:
root=ZFS=rpool/ROOT/pve-1 boot=zfs quiet intel_iommu=on
For AMD:
root=ZFS=rpool/ROOT/pve-1 boot=zfs quiet amd_iommu=on

• Update the Proxmox Kernel Commandline to boot with IOMMU Support
• Enable IOMMU in Proxmox
  

• nano /etc/default/grub
  Comment out #GRUB_CMDLINE_LINUX_DEFAULT="quiet"
  Replace with
  GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on" - For Intel
  GRUB_CMDLINE_LINUX_DEFAULT="quiet AMD_iommu=on" - For AMD
  Save
  update-grub

• Add Kernel Modules

• nano /etc/modules
  vfio
  vfio_iommu_type1
  vfio_pci
  vfio_virqfd

• Blacklist GPU drivers from loading on root Proxmox (may be needed)
  

• nano /etc/modprobe.d/blacklist.conf
  blacklist nvidia
  blacklist nouveau
  blacklist radeon
  blacklist amd

• Ensure your GPU is in a SEPARATE IOMMU  GROUP

• lspci -v
  - Find the group number with your GPU
  find /sys/kernel/iommu_groups/ -type l
  If you do notice that the GPU you’re using is in the same group as another device, run the command below to attempt to isolate the device then reboot Proxmox and run the steps above again to verify.
  echo "options vfio_iommu_type1 allow_unsafe_interrupts=1" > /etc/modprobe.d/iommu_unsafe_interrupts.conf

• Reboot

Virtual Machine Configuration:

• Boot VM normally and setup Remote Desktop
• Shutdown VM
• Ensure BIOS Type is set to OVMF (UEFI)
• Ensure Memory does not have Ballooning Device enabled
• Add the GPU by selecting Add > PCI Device > Select your GPU (Ensure it is the proper IOMMU group as discovered with lspci -v)
• Ensure that All fuctions, ROM-bar, Primary GPU, and PCI-Express are all enabled.
• Change Display Type to NONE
• Log in via Remote Desktop
• Install Graphics Card Drivers
• Reboot.
  
  
• Enjoy.