diff options
| -rw-r--r-- | src/pci_passthrough.md | 203 |
1 files changed, 78 insertions, 125 deletions
diff --git a/src/pci_passthrough.md b/src/pci_passthrough.md index 54bea08..75f169e 100644 --- a/src/pci_passthrough.md +++ b/src/pci_passthrough.md @@ -25,16 +25,14 @@ Once you have enabled IOMMU, you will need to ensure that your PCI slot can actu To list IOMMU groups, you may use this script from the [archwiki](https://wiki.archlinux.org/title/PCI_passthrough_via_OVMF#Ensuring_that_the_groups_are_valid) -``` -\#!/bin/bash -shopt -s nullglob -for g in $(find /sys/kernel/iommu_groups/* -maxdepth 0 -type d | sort -V); do - echo "IOMMU Group ${g##*/}:" - for d in $g/devices/*; do - echo -e "\t$(lspci -nns ${d##*/})" + #!/bin/bash + shopt -s nullglob + for g in $(find /sys/kernel/iommu_groups/* -maxdepth 0 -type d | sort -V); do + echo "IOMMU Group ${g##*/}:" + for d in $g/devices/*; do + echo -e "\t$(lspci -nns ${d##*/})" + done; done; -done; -``` If you see that there PCI devices other than ones that correspond to your graphics card, then try using a different PCI slot. Unfortunately some motherboards don't have any isolated PCI slots, in this case you are out of luck; PCI passthrough will not be possible in this method. @@ -45,17 +43,13 @@ The core principle behind PCI passthrough is that you don't want the kernel from To do this, you want to find out the Device IDs of your PCI device, you can do this using the following command: -``` -lspci -nn -``` + lspci -nn Make sure take notes of all of the devices that appeared within the isolated IOMMU group from the previous stage. If you miss out any PCI devices, this will not work. IDs often look like this: `10de:13c2` Next add the following to your kernel's cmdline arguments. This can usually be found somewhere in your bootloaders settings, for example for GRUB, you can add it to `GRUB_CMDLINE_LINUX_DEFAULT=""` in `/etc/default/grub` -``` -vfio-pci.ids=*id1*,*id2* -``` + vfio-pci.ids=*id1*,*id2* where *id1* and *id2* represent the ids that you collected in the previous step. @@ -64,9 +58,8 @@ Next you need to tell your initramfs (if applicable) to load the vfio modules. T ### Dracut in `/etc/dracut.conf.d/10-vfio.conf` -``` -force_drivers+=" vfio_pci vfio vfio_iommu_type1 vfio_virqfd " -``` + + force_drivers+=" vfio_pci vfio vfio_iommu_type1 vfio_virqfd " then rebuild your initramfs @@ -74,18 +67,16 @@ then rebuild your initramfs ### mkinitpcio in `/etc/mkinitcpio.conf` -``` -MODULES=(... vfio_pci vfio vfio_iommu_type1 vfio_virqfd ...) -``` + + MODULES=(... vfio_pci vfio vfio_iommu_type1 vfio_virqfd ...) then rebuild your initramfs ### booster in `/etc/booster.yaml` -``` -modules_force_load: vfio_pci,vfio,vfio_iommu_type1,vfio_virqfd -``` + + modules_force_load: vfio_pci,vfio,vfio_iommu_type1,vfio_virqfd then rebuild your initramfs @@ -95,60 +86,53 @@ In this guide I will be using qemu from the command line. I feel as if this is t Here is a script that use a to run a linux virtual machine, if you want to copy it blindly and not understand it, then thats alright, but I will include a description for each argument and why its used. -``` -\#!/bin/sh - -ISO="/path/to/your/installer/iso" -ROOT="/path/to/your/virtual/disk" -OVMF_VARS="/path/to/your/ovmf/vars/image" - - -qemu-system-x86_64 \ - -enable-kvm \ - -m 8G \ - -smp 2 \ - -drive if=pflash,format=raw,readonly=on,file=/usr/share/OVMF/OVMF.fd \ - -drive if=pflash,format=raw,file="$OVMF_VARS" \ - -drive if=virtio,file="$ROOT",format=qcow2 \ - -vga none \ - -nographic \ - -monitor stdio \ - -serial none \ - -device vfio-pci,host="06:00.0" \ - -device vfio-pci,host="06:00.1" \ - -device vfio-pci,host="06:00.2" \ - -device vfio-pci,host="06:00.3" \ - -net nic,model=virtio -net user \ - -device ich9-intel-hda,addr=0x1b \ - -device hda-micro,audiodev=hda \ - -device ich9-intel-hda,addr=0x1b \ - -device hda-micro,audiodev=hda \ - -audiodev pa,id=hda,server=unix:$(pactl info | sed -rn 's/Server String: (.*)/\1/p') \ -``` + #!/bin/sh + + ISO="/path/to/your/installer/iso" + ROOT="/path/to/your/virtual/disk" + OVMF_VARS="/path/to/your/ovmf/vars/image" + + + qemu-system-x86_64 \ + -enable-kvm \ + -m 8G \ + -smp 2 \ + -drive if=pflash,format=raw,readonly=on,file=/usr/share/OVMF/OVMF.fd \ + -drive if=pflash,format=raw,file="$OVMF_VARS" \ + -drive if=virtio,file="$ROOT",format=qcow2 \ + -vga none \ + -nographic \ + -monitor stdio \ + -serial none \ + -device vfio-pci,host="06:00.0" \ + -device vfio-pci,host="06:00.1" \ + -device vfio-pci,host="06:00.2" \ + -device vfio-pci,host="06:00.3" \ + -net nic,model=virtio -net user \ + -device ich9-intel-hda,addr=0x1b \ + -device hda-micro,audiodev=hda \ + -device ich9-intel-hda,addr=0x1b \ + -device hda-micro,audiodev=hda \ + -audiodev pa,id=hda,server=unix:$(pactl info | sed -rn 's/Server String: (.*)/\1/p') ### Basic Parameters -``` --enable-kvm -``` + -enable-kvm + This enables [KVM](https://www.linux-kvm.org/page/Main_Page) which improves performance and allows for *almost bare-metal* CPU performance. -``` --m 8G -``` + -m 8G + Allocate 8 Gigabytes of memory to the virtual machine -``` --smp 2 -``` + -smp 2 + Allocate 2 CPU cores to the virtual machine ### OVMF firmware -``` --drive if=pflash,format=raw,readonly=on,file=/usr/share/OVMF/OVMF.fd \ --drive if=pflash,format=raw,file="$OVMF_VARS" \ -``` + -drive if=pflash,format=raw,readonly=on,file=/usr/share/OVMF/OVMF.fd \ + -drive if=pflash,format=raw,file="$OVMF_VARS" \ Most PCI cards require UEFI firmware to function properly. For this reason we will be using the OVMF firmware. You may need to install this on your system, typically the package is called ovmf, edk2-ovmf or something similar. Learn to use your package manager's search functionality to find it. @@ -158,23 +142,17 @@ In my example script, make sure you replace `/path/to/your/ovmf/vars/image` to t ### Storage -``` --drive if=virtio,file="$ROOT",format=qcow2 \ -``` + -drive if=virtio,file="$ROOT",format=qcow2 \ Next you need to create a virtual disk for your virtual machine. You can create this using the following command, replacing `32G` with the desired size of this disk. -``` -qemu-img create -f qcow2 myVirtualDisk.qcow2 32G -``` + qemu-img create -f qcow2 myVirtualDisk.qcow2 32G In this example I use `if=virtio,` which provides better performance than the default drive type, however this will only work with linux guests which have the virtio module. **Remove this if you are using windows in your virtual machine** For your first boot, you may want to add the following before your primary virtual disk: -``` --drive file="$ISO",media=cdrom -``` + -drive file="$ISO",media=cdrom Ensure that you have set the `ISO` and `ROOT` variables appropriately with the paths to the corresponding images. @@ -182,23 +160,19 @@ Ensure that you have set the `ISO` and `ROOT` variables appropriately with the p This next step is to ensure that qemu doesn't create a virtual VGA output for your virtual machine, nor opens a window, allowing this to be run from outside an X11 session -``` --vga none --nographic --monitor stdio --serial none -``` + -vga none + -nographic + -monitor stdio + -serial none ### VFIO passthrough Next you need to find the pci ids for your device. Make sure you include all the relevant ones in your IOMMU group: -``` --device vfio-pci,host="06:00.0" \ --device vfio-pci,host="06:00.1" \ --device vfio-pci,host="06:00.2" \ --device vfio-pci,host="06:00.3" \ -``` + -device vfio-pci,host="06:00.0" \ + -device vfio-pci,host="06:00.1" \ + -device vfio-pci,host="06:00.2" \ + -device vfio-pci,host="06:00.3" \ In this example, my PCI ids start with `06:00`, **make sure you change this for your setup.** @@ -206,19 +180,15 @@ In this example, my PCI ids start with `06:00`, **make sure you change this for Next you want network for your virtual machine: -``` --net nic,model=virtio -net user -``` + -net nic,model=virtio -net user I use the virtio network card, which only works for Linux guests. **If you have a windows guest, do not include this line** this will use the default network card for qemu. ### Audio -``` --device ich9-intel-hda,addr=0x1b \ --device hda-micro,audiodev=hda \ --audiodev pa,id=hda,server=unix:$(pactl info | sed -rn 's/Server String: (.*)/\1/p') \ -``` + -device ich9-intel-hda,addr=0x1b \ + -device hda-micro,audiodev=hda \ + -audiodev pa,id=hda,server=unix:$(pactl info | sed -rn 's/Server String: (.*)/\1/p') \ This creates a audio device for the virtual machine which connects to the currently running pulseaudio session of your client. @@ -232,17 +202,13 @@ Out of all of the methods of passthrough a keyboard and mouse, evdev is probably First you need to identify the input devices that linux creates for your keyboard and mouse. You can list them all using: -``` -ls /dev/input/by-id/ -``` + ls /dev/input/by-id/ Identify the devices that you want to passthrough and find the ones containing `-event`. Then add the follow for each one as an argument to your qemu command. -``` --object input-linux,id=*UNIQUE_ID*,evdev=/dev/input/by-id/*YOUR-DEVICE*,grab_all=on,repeat=on -``` + -object input-linux,id=*UNIQUE_ID*,evdev=/dev/input/by-id/*YOUR-DEVICE*,grab_all=on,repeat=on Then when your virtual machine is running, you will be able to switch to and from the host's control by pressing both left and right ctrl keys at the same time on your keyboard. @@ -250,26 +216,20 @@ Then when your virtual machine is running, you will be able to switch to and fro To do this, I would recommend creating a group named `kvm` and adding your user to it. -``` -groupadd kvm + groupadd kvm -usermod -a -G kvm *username* -``` + usermod -a -G kvm *username* Next you will want to ensure that vfio devices have the correct permissions for the kvm group to use. In `/etc/udev/rules.d/10-vfio.rules`: -``` -SUBSYSTEM=="vfio", GROUP="kvm" -``` + SUBSYSTEM=="vfio", GROUP="kvm" Next you will probably want to increase memory limits for users of the kvm group, to allow them to allocate potentially GB for the virtual machine. To make things easier, you might want to just set this to the maximum number of megabytes available in the system. You can find this out using `free`, for example for a system with 8GB ram, its: `8100452` In `/etc/security/limits.d/99-memlock.conf` write: -``` -@kvm hard memlock 8100452 -@kvm soft memlock 8100452 -``` + @kvm hard memlock 8100452 + @kvm soft memlock 8100452 You may need to reboot for these changes to take effect, especially ones relating to udev rules. @@ -287,29 +247,22 @@ To detect the montors available, use `ddcutil detect`. If this doesn't work, ens To enable the i2c module on load, you may need to add the following to `/etc/modules-load.d/i2c-dev.conf` -``` -i2c-dev -``` + i2c-dev To allow users of the i2c group to control i2c devices add the following to `/etc/udev/rules.d/10-i2c-group.rules`. Make sure you create this group and add your user to it. -``` -KERNEL=="i2c-[0-9]*", GROUP="i2c" -``` + KERNEL=="i2c-[0-9]*", GROUP="i2c" Find out the Display number of your monitor using `ddcutil detect` Next find the available inputs on this monitor: -``` -ddcutil -d $display_number capabilities -``` + ddcutil -d $display_number capabilities Here you should be able to see `Feature: 60 (input source)`. Take note of hexadecimal values. You will be able to switch monitors using: -``` -ddcutil -d $display_number setvcp 60 0x$monitor_input_hex_value -``` + + ddcutil -d $display_number setvcp 60 0x$monitor_input_hex_value I have this bound to a hotkey using `sxhkd` so I can easily switch between inputs without having to reach over to buttons on my monitor.However you can configure this whichever way you want: for example, you can switch to the display output of your passthrough GPU when the virtual machine starts, and back when it shuts down. |