Complete-Single-GPU-Passthrough

You cannot select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

7.2 KiB

Raw Blame History

Automated Setup

Not well-tested. Only supports systemd init system. Installs packages, enables iommu and creates libvirt hooks for now.

Halted till December.

TODO: upstart, openrc, sysv support. Automatic VM creation and passthrough support. Libvirt script for window managers.

make
sudo ./passthrough

OR

sudo make run

Manual Setup

Note: Replace win10 with your virtual machine's name on libvirt hooks and virsh commands.

Enable IOMMU

sudo nano /etc/default/grub

Append iommu=pt, and intel_iommu=on or amd_iommu=on kernel options to GRUB_CMDLINE_LINUX_DEFAULT for your CPU. Then, update grub configuration.

sudo grub-mkconfig -o /boot/grub/grub.cfg

Verify IOMMU

After setting kernel parameter on grub config, reboot your system and verify IOMMU is enabled.

dmesg | grep 'IOMMU enabled'

Install required tools

sudo pacman -S qemu libvirt edk2-ovmf virt-manager dnsmasq ebtables iptables

Enable services

Enable libvirtd service and start default network.

sudo systemctl enable --now libvirtd
sudo virsh net-start default
sudo virsh net-autostart default

Setup Guest OS

Download virtio driver.

Launch virt-manager and create new virtual machine. Check Customize before install on final step.

Overview > Chipset: Q35, Firmware: UEFI
CPUs > CPU model: host-passthrough, CPU topology: Best for your machine :)
SATA Disk > Disk bus: virtio
NIC > Device model: virtio
Add Hardware > CDROM: virtio-win.iso

Begin Installation and install Windows.. Windows can't detect virtio devices, so you need to Load Driver from virtio-disk/amd64/win10 when prompted.

After successful installation, install virtio-drivers from virtio CDROM.

Attach PCI Devices

Remove Tablet, Spice Channel, Video XQL and other applicable devices.

Add Hardware > PCI Devices: GPU and HDMI audio

NVIDIA GPU

Nvidia GPU vBIOS requires patching to work. Though, some NVIDIA GPU works without patching.

Dump GPU vBIOS

Windows > Use GPU-Z
Linux > Try following but doesn't seem to work:

su
echo 1 > /sys/bus/pci/devices/0000:01:00.0/rom
cat /sys/bus/pci/devices/0000:01:00.0/rom > vbios.rom
echo 0 > /sys/bus/pci/devices/0000:01:00.0/rom

Or, you can download vBIOS from TechPowerUp, which might (not) be already patched.

Patching vBIOS

Use Hex Editor and search for string VIDEO, and remove everything before char U, i.e. HEX value 55.

Using Patched vBIOS

sudo virsh edit win10

Search for hostdev. Add

<rom file="path/to/your/patched_vbios.rom"/>

before

<address>

Libvirt Hooks

Libvirt hooks automate the process of running specific tasks during VM state change.

More info at: PassthroughPost

Create Libvirt hook

sudo mkdir /etc/libvirt/hooks
sudo touch /etc/libvirt/hooks/qemu
sudo chmod +x /etc/libvirt/hooks/qemu

/etc/libvirt/hooks/qemu

#!/bin/bash

GUEST_NAME="$1"
HOOK_NAME="$2"
STATE_NAME="$3"
MISC="${@:4}"

BASEDIR="$(dirname $0)"

HOOKPATH="$BASEDIR/qemu.d/$GUEST_NAME/$HOOK_NAME/$STATE_NAME"
set -e # If a script exits with an error, we should as well.

if [ -f "$HOOKPATH" ]; then
  eval \""$HOOKPATH"\" "$@"
elif [ -d "$HOOKPATH" ]; then
  while read file; do
    eval \""$file"\" "$@"
  done <<< "$(find -L "$HOOKPATH" -maxdepth 1 -type f -executable -print;)"
fi

Restart libvirtd

sudo systemctl restart libvirtd

Start Script

sudo mkdir -p /etc/libvirt/hooks/qemu.d/win10/prepare/begin
sudo touch /etc/libvirt/hooks/qemu.d/win10/prepare/begin/start.sh
sudo chmod +x /etc/libvirt/hooks/qemu.d/win10/prepare/begin/start.sh

/etc/libvirt/hooks/qemu.d/win10/prepare/begin/start.sh

#!/bin/bash
set -x

# Stop display manager
systemctl stop display-manager

# Unbind EFI Framebuffer
echo efi-framebuffer.0 > /sys/bus/platform/drivers/efi-framebuffer/unbind

# Unload NVIDIA kernel modules
modprobe -r nvidia_drm nvidia_modeset nvidia_uvm nvidia

# Unload AMD kernel module
# modprobe -r amdgpu

# Detach GPU devices from host
# Use your GPU and HDMI Audio PCI host device
virsh nodedev-detach pci_0000_01_00_0
virsh nodedev-detach pci_0000_01_00_1

# Load vfio module
modprobe vfio-pci

Stop Script

sudo mkdir -p /etc/libvirt/hooks/qemu.d/win10/release/end
sudo touch /etc/libvirt/hooks/qemu.d/win10/release/end/stop.sh
sudo chmod +x /etc/libvirt/hooks/qemu.d/win10/release/end/stop.sh

/etc/libvirt/hooks/qemu.d/win10/release/end/stop.sh

#!/bin/bash
set -x

# Unload vfio module
modprobe -r vfio-pci

# Attach GPU devices to host
# Use your GPU and HDMI Audio PCI host device
virsh nodedev-reattach pci_0000_01_00_0
virsh nodedev-reattach pci_0000_01_00_1

# Rebind framebuffer to host
echo "efi-framebuffer.0" > /sys/bus/platform/drivers/efi-framebuffer/bind

# Load NVIDIA kernel modules
modprobe nvidia_drm
modprobe nvidia_modeset
modprobe nvidia_uvm
modprobe nvidia

# Load AMD kernel module
# modprobe amdgpu

# Restart Display Manager
systemctl start display-manager

Video card driver virtualisation detection
Keyboard/Mouse Passthrough
Audio Passthrough
Troubleshooting

See Also

Single GPU Passthrough by joeknock90
Single GPU Passthrough by YuriAlek
ArchLinux PCI Passthrough
Gentoo GPU Passthrough