Archives
/
Complete-Single-GPU-Passthrough
mirror of https://github.com/QaidVoid/Complete-Single-GPU-Passthrough
2
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
You cannot select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
master
Branches Tags
${ item.name }
Create tag ${ searchTerm }
Create branch ${ searchTerm }
from 'b0fdd478a7'
${ noResults }
Complete-Single-GPU-Passthr.../README.md

9.2 KiB
Raw Blame History

Enable IOMMU

Set the kernel paramater depending on your CPU.
For GRUB user, edit grub configuration.

/etc/default/grub
GRUB_CMDLINE_LINUX_DEFAULT="... intel_iommu=on iommu=pt ..."
OR
GRUB_CMDLINE_LINUX_DEFAULT="... amd_iommu=on iommu=pt ..."
Generate grub.cfg 
grub-mkconfig -o /boot/grub/grub.cfg

Reboot your system for the changes to take effect.

Verify IOMMU

If you don't see any output when running following command, IOMMU is not functioning.

dmesg | grep 'IOMMU enabled'

Install required tools

Gentoo Linux 
emerge -av qemu virt-manager libvirt ebtables dnsmasq
Arch Linux 
pacman -S qemu libvirt edk2-ovmf virt-manager dnsmasq ebtables
Fedora 
dnf install @virtualization

Enable required services

SystemD 
systemctl enable --now libvirtd
OpenRC 
rc-update add libvirtd default
rc-service libvirtd start

Sometimes, you might need to start default network manually.

virsh net-start default
virsh net-autostart default

Setup Guest OS

NOTE: You should replace win10 with your VM's name where applicable
Download virtio driver.
Launch virt-manager and create a new virtual machine. Select Customize before install on Final Step.
In Overview section, set Chipset to Q35, and Firmware to UEFI
In CPUs section, set CPU model to host-passthrough, and CPU Topology to whatever fits your system.
For SATA disk of VM, set Disk Bus to virtio.
In NIC section, set Device Model to virtio
Add Hardware > CDROM: virtio-win.iso
Now, Begin Installation. Windows can't detect the virtio disk, so you need to Load Driver and select virtio-iso/amd64/win10 when prompted.
After successful installation of Windows, install virtio drivers from virtio CDROM.

Attaching PCI devices

Remove Channel Spice, Display Spice, Video XQL, Sound ich* and other unnecessary devices.
Now, click on Add Hardware, select PCI Devices and add the PCI Host devices for your GPU's VGA and HDMI Audio
Some GPU vBIOS needs to be patched for UEFI Support.
----- TODO: vBIOS patching ------
To use patched vBIOS, edit VM's configuration to include patched vBIOS inside hostdev block of VGA

mkdir /etc/libvirt/hooks
touch /etc/libvirt/hooks/qemu
chmod +x /etc/libvirt/hooks/qemu
virsh edit win10 
...
<hostdev mode='subsystem' type='pci' managed='yes'>
  <source>
    ...
  </source>
  <rom file='/home/me/patched.rom'/>
  ...
</hostdev>
...

Libvirt Hooks

Libvirt hooks automate the process of running specific tasks during VM state change.
More info at: PassthroughPost

Create Libvirt Hook 
mkdir /etc/libvirt/hooks
touch /etc/libvirt/hooks/qemu
chmod +x /etc/libvirt/hooks/qemu
/etc/libvirt/hooks/qemu 
#!/bin/bash

GUEST_NAME="$1"
HOOK_NAME="$2"
STATE_NAME="$3"
MISC="${@:4}"

BASEDIR="$(dirname $0)"

HOOKPATH="$BASEDIR/qemu.d/$GUEST_NAME/$HOOK_NAME/$STATE_NAME"
set -e # If a script exits with an error, we should as well.

if [ -f "$HOOKPATH" ]; then
eval \""$HOOKPATH"\" "$@"
elif [ -d "$HOOKPATH" ]; then
while read file; do
  eval \""$file"\" "$@"
done <<< "$(find -L "$HOOKPATH" -maxdepth 1 -type f -executable -print;)"
fi
Create Start Script 
mkdir -p /etc/libvirt/hooks/qemu.d/win10/prepare/begin
touch /etc/libvirt/hooks/qemu.d/win10/prepare/begin/start.sh
chmod +x /etc/libvirt/hooks/qemu.d/win10/prepare/begin/start.sh
/etc/libvirt/hooks/qemu.d/win10/prepare/begin/start.sh 
#!/bin/bash
set -x

# Stop display manager
systemctl stop display-manager
# rc-service display-manager stop

# Unbind EFI Framebuffer
echo efi-framebuffer.0 > /sys/bus/platform/drivers/efi-framebuffer/unbind

# Unload NVIDIA kernel modules
modprobe -r nvidia_drm nvidia_modeset nvidia_uvm nvidia

# Unload AMD kernel module
# modprobe -r amdgpu

# Detach GPU devices from host
# Use your GPU and HDMI Audio PCI host device
virsh nodedev-detach pci_0000_01_00_0
virsh nodedev-detach pci_0000_01_00_1

# Load vfio module
modprobe vfio-pci
Create Stop Script 
mkdir -p /etc/libvirt/hooks/qemu.d/win10/release/end
touch /etc/libvirt/hooks/qemu.d/win10/release/end/stop.sh
chmod +x /etc/libvirt/hooks/qemu.d/win10/release/end/stop.sh
/etc/libvirt/hooks/qemu.d/win10/release/end/stop.sh 
#!/bin/bash
set -x

# Unload vfio module
modprobe -r vfio-pci

# Attach GPU devices to host
# Use your GPU and HDMI Audio PCI host device
virsh nodedev-reattach pci_0000_01_00_0
virsh nodedev-reattach pci_0000_01_00_1

# Rebind framebuffer to host
echo "efi-framebuffer.0" > /sys/bus/platform/drivers/efi-framebuffer/bind

# Load NVIDIA kernel modules
modprobe nvidia_drm
modprobe nvidia_modeset
modprobe nvidia_uvm
modprobe nvidia

# Load AMD kernel module
# modprobe amdgpu

# Restart Display Manager
systemctl start display-manager
# rc-service display-manager start

Keyboard/Mouse Passthrough

Modify libvirt configuration of your VM. Change first line to:

virsh edit win10 
<domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>

Find your keyboard and mouse devices in /dev/input/by-id. You'd generally use the devices ending with event-kbd and event-mouse. And the devices in your configuration right before closing </domain> tag.
Replace MOUSE_NAME and KEYBOARD_NAME with your device id.

virsh edit win10 
...
  <qemu:commandline>
    <qemu:arg value='-object'/>
    <qemu:arg value='input-linux,id=mouse1,evdev=/dev/input/by-id/MOUSE_NAME'/>
    <qemu:arg value='-object'/>
    <qemu:arg value='input-linux,id=kbd1,evdev=/dev/input/by-id/KEYBOARD_NAME,grab_all=on,repeat=on'/>
  </qemu:commandline>
</domain>

You need to include these devices in your qemu config.

/etc/libvirt/qemu.conf 
...
user = "YOUR_USERNAME"
group = "kvm"
...
cgroup_device_acl = [
    "/dev/input/by-id/KEYBOARD_NAME",
    "/dev/input/by-id/MOUSE_NAME",
    "/dev/null", "/dev/full", "/dev/zero",
    "/dev/random", "/dev/urandom",
    "/dev/ptmx", "/dev/kvm", "/dev/kqemu",
    "/dev/rtc","/dev/hpet", "/dev/sev"
]
...

Also, switch from PS/2 devices to virtio devices. Add the devices inside block

virsh edit win10 
...
<devices>
  ...
  <input type='mouse' bus='virtio'/>
  <input type='keyboard' bus='virtio'/>
  ...
</devices>
...

Audio Passthrough

VM's audio can be routed to the host. You need Pulseaudio.
Modify the libvirt configuration of your VM.

virsh edit win10 
...
  <qemu:commandline>
    ...
    <qemu:arg value="-device"/>
    <qemu:arg value="ich9-intel-hda,bus=pcie.0,addr=0x1b"/>
    <qemu:arg value="-device"/>
    <qemu:arg value="hda-micro,audiodev=hda"/>
    <qemu:arg value="-audiodev"/>
    <qemu:arg value="pa,id=hda,server=/run/user/1000/pulse/native"/>
  </qemu:commandline>
</devices>

Video card driver virtualisation detection

Video Card drivers refuse to run in Virtual Machine, so you need to spoof Hyper-V Vendor ID.

virsh edit win10 
...
<features>
  ...
  <hyperv>
    ...
    <vendor_id state='on' value='whatever'/>
    ...
  </hyperv>
  ...
</features>
...

Nvidia guest drivers also require hiding the KVM CPU leaf:

virsh edit win10 
...
<features>
  ...
  <kvm>
    <hidden state='on'/>
  </kvm>
  ...
</features>
...

See Also

Single GPU Passthrough by joeknock90
Single GPU Passthrough by YuriAlek
ArchLinux PCI Passthrough
Gentoo GPU Passthrough