Binding NIC drivers

As DPDK uses its own poll-mode drivers in userspace instead of traditional kernel drivers, the kernel needs to be told to use a different, pass-through style driver for the devices: VFIO (Virtual Functio I/O) or UIO (Userspace I/O).

Between the two, use VFIO if you can. UIO was never designed for use with DMA-capable devices and has no provisions to protect against userspace corrupting arbitrary kernel memory. Additionally uio_pci_generic only supports legacy interrupts (as opposed to MSI/MSI-X), which means it cannot be used with eg SR-IOV and virtual hosts at all.

There are two different tools for binding drivers: driverctl which is a generic tool for persistently configuring alternative device drivers, and dpdk_nic_bind which is a DPDK-specific tool and whose changes do not persist across reboots.

These instructions apply to all physical and emulated NIC’s, but see Common gotchas for virtual NICs such as virtio-net.

Changed in version 16.07: In DPDK 16.07, the dpdk_nic_bind utility was renamed to dpdk-devbind.

VFIO

Of the two, VFIO is the more robust and secure option. Unfortunately it also has additional requirements for both hardware and system setup: IOMMU must be present and enabled on the system first.

With driverctl:

# driverctl -v list-devices | grep -i net
# driverctl set-override <pci-slot> vfio-pci

With dpdk_nic_bind (DPDK <= 16.04):

# modprobe vfio-pci
# dpdk_nic_bind --status
# dpdk_nic_bind --bind=vfio-pci <pci-slot>

With dpdk-devbind (DPDK >= 16.07:

# modprobe vfio-pci
# dpdk-devbind --status
# dpdk-devbind --bind=vfio-pci <pci-slot>

VFIO no-IOMMU

Linux kernel >= 4.5 and DPDK 16.04 support a new VFIO mode which allows VFIO to be used without IOMMU. While this is just as unsafe as using UIO, it does make it possible to use VFIO in situations where IOMMU is not available and uio_pci_generic does not work either, such as virtual hosts.

The no-IOMMU mode is off by default and must be specifically enabled:

# modprobe vfio-pci
# echo 1 > /sys/module/vfio/parameters/enable_unsafe_noiommu_mode

To permanently enable the no-IOMMU mode, add it to modprobe.d:

echo "options vfio enable_unsafe_noiommu_mode=1" > /etc/modprobe.d/vfio-noiommu.conf

Otherwise the binding is identical to normal VFIO. Note that the no-IOMMU mode will taint the kernel.

Changed in version 16.04: This option is new as of DPDK 16.04 and Linux kernel 4.5.

UIO

UIO is not safe with DMA-capable hardware, which all modern NICs are. However since UIO doesn’t need any further setup, it can be nice for a quick spin on non-critical test system where data integrity is not an issue. Note that uio_pci_generic only supports legacy interrupts so it is not usable with SR-IOV and virtual hosts which require MSI/MSI-X interrupts.

With driverctl:

# driverctl -v list-devices | grep -i net
# driverctl set-override <pci-slot> uio_pci_generic

With dpdk_nic_bind (DPDK <= 16.04):

# modprobe uio_pci_generic
# dpdk_nic_bind --status
# dpdk_nic_bind --bind=uio_pci_generic <pci-slot>

With dpdk-devbind (DPDK >= 16.07):

# modprobe uio_pci_generic
# dpdk-devbind --status
# dpdk-devbind --bind=uio_pci_generic <pci-slot>