src/libixy-vfio.h

VFIO entry-point declarations.

Walkthrough, interview notes & deep dive

Public Interface. This header defines the public entry-point declarations for the ixy VFIO (Virtual Function I/O) backend. It abstracts the Linux kernel's VFIO interface, allowing a userspace driver to interact with PCI hardware. For an engineer working on Solarflare or AMD NICs, understanding this layer is critical because it replaces the older, less secure UIO approach with a robust, IOMMU-protected memory model.

Initialization Flow. The initialization starts with vfio_init, which takes a PCI address string and returns a file descriptor. Internally, this involves a sequence of opening the /dev/vfio/vfio container, finding the device's IOMMU group, and finally obtaining the device-specific file descriptor. Once initialized, vfio_enable_dma is called to set the Bus Master bit in the PCI configuration space.

int vfio_init(const char* pci_addr);
void vfio_enable_dma(int device_fd);
uint8_t* vfio_map_region(int vfio_fd, int region_index);

Memory Management. To access device registers, vfio_map_region maps PCI BARs into the process's virtual address space. For the high-performance data path, vfio_map_dma is used to register memory for packet buffers. This function informs the kernel to "pin" these pages in RAM and map them through the IOMMU using the VFIO_IOMMU_MAP_DMA ioctl.

uint64_t vfio_map_dma(void* vaddr, uint32_t size);
uint64_t vfio_unmap_dma(int fd, uint64_t iova, uint32_t size);

IOMMU and DMA Safety. The primary advantage of VFIO is its use of the IOMMU (Input-Output Memory Management Unit). Unlike raw physical addresses, the device uses an I/O Virtual Address (IOVA) returned by vfio_map_dma. The IOMMU acts as a hardware-level page table, translating IOVAs to physical addresses and ensuring the NIC cannot access memory outside of what has been explicitly mapped. This provides critical isolation between the userspace driver and the rest of the system.

Interrupt Handling. While ixy often operates in polling mode, this header provides functions like vfio_enable_msix and vfio_setup_interrupt for MSI-X support. These are used to handle asynchronous events or to implement hybrid interrupt/polling schemes (like NAPI in the kernel).

int vfio_enable_msix(int device_fd, uint32_t interrupt_vector);
int vfio_setup_interrupt(int device_fd);

Interview angles

What is the primary difference between VFIO and UIO? VFIO uses the IOMMU to provide memory protection and isolation, whereas UIO uses raw physical addresses and requires the driver to be fully trusted.
Why is an IOVA used instead of a physical address? An IOVA is a virtual address for the hardware; the IOMMU translates it to a physical address, allowing the kernel to restrict device access to specific memory pages.
Why must we register DMA memory with VFIO? The IOMMU blocks all DMA by default for security. vfio_map_dma pins the memory (preventing it from being swapped) and programs the IOMMU to allow hardware access to those specific pages.

Going deeper

uint64_t vfio_map_dma(void* vaddr, uint32_t size);
uint64_t vfio_unmap_dma(int fd, uint64_t iova, uint32_t size);

The API exhibits a curious asymmetry: vfio_map_dma lacks a file descriptor, implying the implementation tracks a global VFIO container. This restricts the driver to a single IOMMU domain. Contrast this with vfio_map_region, which requires an explicit vfio_fd to map device BARs. The use of uint64_t for unmap return values—rather than a standard int—suggests a pattern of using the full 64-bit space to tunnel error codes or confirm the original IOVA, avoiding dependency on errno in the fast path.

Harder interview questions

What happens if vfio_init is called for a device in an IOMMU group where another device is still bound to a kernel driver? The call will fail. VFIO requires all devices in a "non-viable" group to be detached from host drivers to prevent side-channel DMA attacks between the userspace driver and the kernel.
How does vfio_map_dma handle memory pinning? It doesn't just map; it triggers the kernel to pin pages and update IOMMU tables. If the user passes a vaddr from standard malloc, the kernel may fail the request if the mapping exceeds the RLIMIT_MEMLOCK (locked-in-memory size) system limit.

Gotchas

The uint32_t size parameter in DMA functions creates a silent 4GB limit. In systems with massive mempools, passing a size_t larger than 32 bits will result in truncation and partial mappings.
vfio_map_region returns uint8_t* but expects a check against MAP_FAILED. Because MAP_FAILED is (void *)-1, a simple if (!ptr) null-check will fail to catch errors, leading to immediate segfaults on BAR access.

From ixy to a production driver

Production shape ixy's VFIO header exposes the pieces a userspace driver must touch directly: VFIO_GET_API_VERSION, VFIO_CHECK_EXTENSION, group/device discovery, VFIO_GROUP_GET_DEVICE_FD, VFIO_DEVICE_GET_REGION_INFO for BAR mmap, VFIO_IOMMU_MAP_DMA / VFIO_IOMMU_UNMAP_DMA for IOVA translation, VFIO_DEVICE_SET_IRQS for MSI/MSI-X, plus the PCI command register Bus Master Enable bit. A real in-kernel ixgbe.ko driver hides that behind subsystems: it enables PCI resources, allocates descriptor rings with dma_alloc_coherent, maps packet buffers with dma_map_single, requests MSI-X vectors, wires them to request_irq, and switches receive work into NAPI. Cleanup is tied to devm_* and remove paths rather than public helper calls.

DPDK contrast DPDK is closer to ixy because poll-mode drivers can also sit on vfio-pci, but VFIO details live in EAL. EAL owns containers and groups, maps huge-page memory, chooses IOVA mode, records mappings for hotplug/remap cases, and gives PMDs stable mbuf memory. In IOVA=VA mode the device-visible address can follow the process VA layout; DPDK uses sentinels such as RTE_BAD_IOVA (-1) when no IOVA is available yet. Interrupts go through rte_intr_*, which ultimately programs VFIO_DEVICE_SET_IRQS with eventfds.

What ixy leaves out

It assumes a simple VFIO world: one process, effectively one global container/IOMMU domain, and no serious SR-IOV, multi-device, hotplug, or VFIO group viability recovery story.
DMA mapping takes uint32_t size, fine for teaching rings and packet pools but capping one map at 4 GiB.
Page pinning from VFIO_IOMMU_MAP_DMA can hit RLIMIT_MEMLOCK; DPDK reduces that pain with pre-reserved hugepages and centralized bookkeeping.
Error handling is thin: production code retries, unwinds partial setup, copes with failed ioctls, FLR, device removal, and IOMMU dirty-page tracking for live migration.
Interrupt support proves MSI/MSI-X through eventfd/epoll, but ixy mainly polls. ixgbe uses per-queue MSI-X, RSS-aware affinity, interrupt moderation, and NAPI; production drivers may also use hardware features ixy ignores, such as DCA.

Why this is acceptable ixy is deliberately small so the reader can see the contract: Bus Master lets the NIC issue DMA, BAR mapping exposes registers, VFIO pins memory and installs IOVA translations, and MSI/MSI-X turns device writes into host events. In an interview, the important gap is knowing which parts are educational shortcuts and which parts production software must own for isolation, scale, failure recovery, and operability.

Sources

Source

filesrc/libixy-vfio.h

#ifndef LIBIXY_VFIO_H
#define LIBIXY_VFIO_H

#include <stdint.h>

// enables DMA on a VFIO device
void vfio_enable_dma(int device_fd);

// initializes the IOMMU for the device. returns the devices file descriptor or
// -1 on error
int vfio_init(const char* pci_addr);

int vfio_enable_msi(int device_fd);

int vfio_disable_msi(int device_fd);

int vfio_enable_msix(int device_fd, uint32_t interrupt_vector);

int vfio_disable_msix(int device_fd);

int vfio_setup_interrupt(int device_fd);

int vfio_epoll_wait(int epoll_fd, int maxevents, int timeout);

int vfio_epoll_ctl(int event_fd);

// returns a uint8_t pointer to the MMAPED region or MAP_FAILED if failed.
// region_index is to be taken from linux/vfio.h
uint8_t* vfio_map_region(int vfio_fd, int region_index);

// returns iova (physical address of the DMA memory from device view) on success
// or -1 else
uint64_t vfio_map_dma(void* vaddr, uint32_t size);

// unmaps previously mapped DMA region. returns 0 on success
uint64_t vfio_unmap_dma(int fd, uint64_t iova, uint32_t size);

#endif //LIBIXY_VFIO_H