Binding NIC drivers¶
As DPDK uses its own poll-mode drivers in userspace instead of traditional kernel drivers, the kernel needs to be told to use a different, pass-through style driver for the devices: VFIO (Virtual Functio I/O) or UIO (Userspace I/O).
Between the two, use VFIO if you can.
UIO was never designed for use with DMA-capable devices and has no provisions
to protect against userspace corrupting arbitrary kernel memory. Additionally
uio_pci_generic
only supports legacy interrupts (as opposed to MSI/MSI-X),
which means it cannot be used with eg SR-IOV and virtual hosts at all.
There are two different tools for binding drivers: driverctl
which
is a generic tool for persistently configuring alternative device drivers,
and dpdk_nic_bind
which is a DPDK-specific tool and whose
changes do not persist across reboots.
These instructions apply to all physical and emulated NIC’s, but see Common gotchas for virtual NICs such as virtio-net.
Changed in version 16.07: In DPDK 16.07, the dpdk_nic_bind utility
was renamed to
dpdk-devbind
.
VFIO¶
Of the two, VFIO is the more robust and secure option. Unfortunately it also has additional requirements for both hardware and system setup: IOMMU must be present and enabled on the system first.
With driverctl
:
# driverctl -v list-devices | grep -i net
# driverctl set-override <pci-slot> vfio-pci
With dpdk_nic_bind
(DPDK <= 16.04):
# modprobe vfio-pci
# dpdk_nic_bind --status
# dpdk_nic_bind --bind=vfio-pci <pci-slot>
With dpdk-devbind
(DPDK >= 16.07:
# modprobe vfio-pci
# dpdk-devbind --status
# dpdk-devbind --bind=vfio-pci <pci-slot>
VFIO no-IOMMU¶
Linux kernel >= 4.5 and DPDK 16.04 support a new VFIO mode which allows VFIO to be used without IOMMU. While this is just as unsafe as using UIO, it does make it possible to use VFIO in situations where IOMMU is not available and uio_pci_generic does not work either, such as virtual hosts.
The no-IOMMU mode is off by default and must be specifically enabled:
# modprobe vfio-pci
# echo 1 > /sys/module/vfio/parameters/enable_unsafe_noiommu_mode
To permanently enable the no-IOMMU mode, add it to modprobe.d:
echo "options vfio enable_unsafe_noiommu_mode=1" > /etc/modprobe.d/vfio-noiommu.conf
Otherwise the binding is identical to normal VFIO. Note that the no-IOMMU mode will taint the kernel.
Changed in version 16.04: This option is new as of DPDK 16.04 and Linux kernel 4.5.
UIO¶
UIO is not safe with DMA-capable hardware, which all modern NICs are.
However since UIO doesn’t need any further setup, it can be nice for
a quick spin on non-critical test system where data integrity is not
an issue. Note that uio_pci_generic
only supports legacy interrupts
so it is not usable with SR-IOV and virtual hosts which require
MSI/MSI-X interrupts.
With driverctl
:
# driverctl -v list-devices | grep -i net
# driverctl set-override <pci-slot> uio_pci_generic
With dpdk_nic_bind
(DPDK <= 16.04):
# modprobe uio_pci_generic
# dpdk_nic_bind --status
# dpdk_nic_bind --bind=uio_pci_generic <pci-slot>
With dpdk-devbind
(DPDK >= 16.07):
# modprobe uio_pci_generic
# dpdk-devbind --status
# dpdk-devbind --bind=uio_pci_generic <pci-slot>