Complete-Single-GPU-Passthrough

You cannot select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

6.3 KiB

Raw Blame History

Enable IOMMU

Set the kernel paramater depending on your CPU.
For GRUB user, edit grub configuration.

/etc/default/grub
`GRUB_CMDLINE_LINUX_DEFAULT="... intel_iommu=on iommu=pt ..."`
OR
`GRUB_CMDLINE_LINUX_DEFAULT="... amd_iommu=on iommu=pt ..."`
Generate grub.cfg

grub-mkconfig -o /boot/grub/grub.cfg

Reboot your system for the changes to take effect.

Verify IOMMU

If you don't see any output when running following command, IOMMU is not functioning.

dmesg | grep 'IOMMU enabled'

Install required tools

Gentoo Linux

emerge -av qemu virt-manager libvirt ebtables dnsmasq

Arch Linux

pacman -S qemu libvirt edk2-ovmf virt-manager dnsmasq ebtables

Fedora

dnf install @virtualization

Enable required services

SystemD

systemctl enable --now libvirtd

OpenRC

rc-update add libvirtd default
rc-service libvirtd start

Sometimes, you might need to start default network manually.

virsh net-start default
virsh net-autostart default

Setup Guest OS

NOTE: You should replace win10 with your VM's name where applicable
Download virtio driver.
Launch virt-manager and create a new virtual machine. Select Customize before install on Final Step.
In Overview section, set Chipset to Q35, and Firmware to UEFI
In CPUs section, set CPU model to host-passthrough, and CPU Topology to whatever fits your system.
For SATA disk of VM, set Disk Bus to virtio.
In NIC section, set Device Model to virtio
Add Hardware > CDROM: virtio-win.iso
Now, Begin Installation. Windows can't detect the virtio disk, so you need to Load Driver and select virtio-iso/amd64/win10 when prompted.
After successful installation of Windows, install virtio drivers from virtio CDROM.

Attaching PCI devices

Remove Channel Spice, Display Spice, Video XQL, Sound ich* and other unnecessary devices.
Now, click on Add Hardware, select PCI Devices and add the PCI Host devices for your GPU's VGA and HDMI Audio
Some GPU vBIOS needs to be patched for UEFI Support.
----- TODO: vBIOS patching ------
To use patched vBIOS, edit VM's configuration to include patched vBIOS inside hostdev block of VGA

mkdir /etc/libvirt/hooks
touch /etc/libvirt/hooks/qemu
chmod +x /etc/libvirt/hooks/qemu

virsh edit win10
`... <hostdev mode='subsystem' type='pci' managed='yes'> <source> ... </source> <rom file='/home/me/patched.rom'/> ... </hostdev> ...`

Libvirt Hooks

Libvirt hooks automate the process of running specific tasks during VM state change.
More info at: PassthroughPost

Create Libvirt Hook

mkdir /etc/libvirt/hooks
touch /etc/libvirt/hooks/qemu
chmod +x /etc/libvirt/hooks/qemu

/etc/libvirt/hooks/qemu

/etc/libvirt/hooks/qemu
`#!/bin/bash GUEST_NAME="$1" HOOK_NAME="$2" STATE_NAME="$3" MISC="${@:4}" BASEDIR="$(dirname $0)" HOOKPATH="$BASEDIR/qemu.d/$GUEST_NAME/$HOOK_NAME/$STATE_NAME" set -e # If a script exits with an error, we should as well. if [ -f "$HOOKPATH" ]; then eval \""$HOOKPATH"\" "$@" elif [ -d "$HOOKPATH" ]; then while read file; do eval \""$file"\" "$@" done <<< "$(find -L "$HOOKPATH" -maxdepth 1 -type f -executable -print;)" fi`

#!/bin/bash

GUEST_NAME="$1"
HOOK_NAME="$2"
STATE_NAME="$3"
MISC="${@:4}"

BASEDIR="$(dirname $0)"

HOOKPATH="$BASEDIR/qemu.d/$GUEST_NAME/$HOOK_NAME/$STATE_NAME"
set -e # If a script exits with an error, we should as well.

if [ -f "$HOOKPATH" ]; then
eval \""$HOOKPATH"\" "$@"
elif [ -d "$HOOKPATH" ]; then
while read file; do
  eval \""$file"\" "$@"
done <<< "$(find -L "$HOOKPATH" -maxdepth 1 -type f -executable -print;)"
fi

Create Start Script

mkdir -p /etc/libvirt/hooks/qemu.d/win10/prepare/begin
touch /etc/libvirt/hooks/qemu.d/win10/prepare/begin/start.sh
chmod +x /etc/libvirt/hooks/qemu.d/win10/prepare/begin/start.sh

/etc/libvirt/hooks/qemu.d/win10/prepare/begin/start.sh

/etc/libvirt/hooks/qemu.d/win10/prepare/begin/start.sh
#!/bin/bash set -x # Stop display manager systemctl stop display-manager # rc-service display-manager stop # Unbind EFI Framebuffer echo efi-framebuffer.0 > /sys/bus/platform/drivers/efi-framebuffer/unbind # Unload NVIDIA kernel modules modprobe -r nvidia_drm nvidia_modeset nvidia_uvm nvidia # Unload AMD kernel module # modprobe -r amdgpu # Detach GPU devices from host # Use your GPU and HDMI Audio PCI host device virsh nodedev-detach pci_0000_01_00_0 virsh nodedev-detach pci_0000_01_00_1 # Load vfio module modprobe vfio-pci

#!/bin/bash
set -x

# Stop display manager
systemctl stop display-manager
# rc-service display-manager stop

# Unbind EFI Framebuffer
echo efi-framebuffer.0 > /sys/bus/platform/drivers/efi-framebuffer/unbind

# Unload NVIDIA kernel modules
modprobe -r nvidia_drm nvidia_modeset nvidia_uvm nvidia

# Unload AMD kernel module
# modprobe -r amdgpu

# Detach GPU devices from host
# Use your GPU and HDMI Audio PCI host device
virsh nodedev-detach pci_0000_01_00_0
virsh nodedev-detach pci_0000_01_00_1

# Load vfio module
modprobe vfio-pci

Create Stop Script

mkdir -p /etc/libvirt/hooks/qemu.d/win10/release/end
touch /etc/libvirt/hooks/qemu.d/win10/release/end/stop.sh
chmod +x /etc/libvirt/hooks/qemu.d/win10/release/end/stop.sh

/etc/libvirt/hooks/qemu.d/win10/release/end/stop.sh

/etc/libvirt/hooks/qemu.d/win10/release/end/stop.sh
#!/bin/bash set -x # Unload vfio module modprobe -r vfio-pci # Attach GPU devices to host # Use your GPU and HDMI Audio PCI host device virsh nodedev-reattach pci_0000_01_00_0 virsh nodedev-reattach pci_0000_01_00_1 # Rebind framebuffer to host echo "efi-framebuffer.0" > /sys/bus/platform/drivers/efi-framebuffer/bind # Load NVIDIA kernel modules modprobe nvidia_drm modprobe nvidia_modeset modprobe nvidia_uvm modprobe nvidia # Load AMD kernel module # modprobe amdgpu # Restart Display Manager systemctl start display-manager # rc-service display-manager start

#!/bin/bash
set -x

# Unload vfio module
modprobe -r vfio-pci

# Attach GPU devices to host
# Use your GPU and HDMI Audio PCI host device
virsh nodedev-reattach pci_0000_01_00_0
virsh nodedev-reattach pci_0000_01_00_1

# Rebind framebuffer to host
echo "efi-framebuffer.0" > /sys/bus/platform/drivers/efi-framebuffer/bind

# Load NVIDIA kernel modules
modprobe nvidia_drm
modprobe nvidia_modeset
modprobe nvidia_uvm
modprobe nvidia

# Load AMD kernel module
# modprobe amdgpu

# Restart Display Manager
systemctl start display-manager
# rc-service display-manager start

6.3 KiB Raw Blame History