Running your own self-hosted AI server (Ollama, Open WebUI, vLLM) alongside media services like Plex or Jellyfin on Proxmox VE is one of the smartest homelab upgrades you can make in 2026. However, passing a dedicated NVIDIA RTX or AMD Radeon GPU directly into a Virtual Machine (VM) or LXC container often triggers notorious roadblocks: Windows VM Error Code 43, host screen freezes, or broken IOMMU groups.

This guide walks through a clean, production-tested PCIe VFIO GPU Passthrough configuration on Proxmox VE 8.x and 9.x that isolates your GPU cleanly without crashing the hypervisor.

⚡ Planning Your AI Server Hardware? Calculate how much VRAM your quantized models need before configuring passthrough using our Interactive AI Model VRAM Calculator on NextByte Tech →

1. VM (VFIO) vs. Unprivileged LXC Container GPU Passthrough

Feature / Metric Full VM Passthrough (VFIO-PCI) LXC Container Passthrough (Bind Mount)
Host GPU Sharing Exclusive to 1 VM at a time Can be shared across multiple LXC containers (Ollama + Plex)
NVIDIA Driver Location Installed inside the Guest OS only Installed on Proxmox Host & matched inside LXC
Memory & Compute Overhead ~2–3% hypervisor overhead, fixed RAM allocation Near-zero (~0.2%), dynamic RAM sharing
Kernel Isolation & Security ⭐⭐⭐⭐⭐ Hardware-level isolation ⭐⭐⭐ Shared host Linux kernel

2. Step 1: Enable IOMMU in BIOS and Proxmox GRUB / systemd-boot

First, enable Intel VT-d or AMD IOMMU (SVM) plus Above 4G Decoding and Resizable BAR in your motherboard BIOS. Then SSH into your Proxmox node and edit `/etc/default/grub`:

Terminal / Script
nano /etc/default/grub # For Intel CPUs: GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on iommu=pt" # For AMD Ryzen / EPYC CPUs: GRUB_CMDLINE_LINUX_DEFAULT="quiet amd_iommu=on iommu=pt"

Update GRUB and verify that IOMMU remapping is active after rebooting:

Terminal / Script
update-grub reboot # After reboot, verify: dmesg | grep -e DMAR -e IOMMU -e AMD-Vi

3. Step 2: Load VFIO Modules and Bind GPU PCIe IDs

Add the four mandatory VFIO kernel modules to `/etc/modules`:

⚡
Interactive Tool

PC Bottleneck & PSU Wattage Calculator 2026

Calculate peak wattage draw, 80 Plus power supply headroom, and 12V-2x6 cable compatibility before buying.

Calculate PSU Match →
Terminal / Script
cat <<EOF >> /etc/modules vfio vfio_iommu_type1 vfio_pci vfio_virqfd EOF

Next, find both the VGA Controller ID and the Audio Controller ID of your GPU:

Terminal / Script
lspci -nn | grep -i nvidia # Example output: # 01:00.0 VGA compatible controller [0300]: NVIDIA Corporation AD104 [GeForce RTX 4070] [10de:2786] # 01:00.1 Audio device [0403]: NVIDIA Corporation AD104 High Definition Audio [10de:22bc]

Bind those vendor IDs to `vfio-pci` and blacklist host drivers so Proxmox does not grab the GPU during boot:

Terminal / Script
echo "options vfio-pci ids=10de:2786,10de:22bc disable_vga=1" > /etc/modprobe.d/vfio.conf cat <<EOF > /etc/modprobe.d/blacklist-gpu.conf blacklist nouveau blacklist nvidia blacklist nvidiafb blacklist nvidia_drm EOF update-initramfs -u -k all reboot

4. Step 3: Attach the GPU to Your Ubuntu AI VM (`q35` + `OVMF`)

When creating your VM in the Proxmox Web UI, use these exact hardware settings to prevent `Code 43` and PCIe bandwidth throttling:

  • Machine Type: q35 (Required for PCIe passthrough)
  • BIOS: OVMF (UEFI) with an EFI Disk (Disable Secure Boot inside the guest BIOS if installing unsigned Linux DKMS modules)
  • CPU Type: host (Exposes AVX2/AVX-512 instructions to Ollama and llama.cpp)
  • Add PCI Device: Select your GPU, check All Functions, ROM-Bar, and PCI-Express.

Boot your VM, install the official NVIDIA CUDA Toolkit or `nvidia-driver-565`, and run `nvidia-smi`—your GPU will now operate at full PCIe Gen4/Gen5 bandwidth with zero host conflicts.

About the Author: Binod Bhatt is a systems developer and hardware researcher at NextByte Tech, specializing in Linux kernel benchmarks, local AI deployments, and interactive hardware optimization tools.