src/driver/ixgbe.h

ixgbe device struct and the public ixgbe_* entry points.

Walkthrough, interview notes & deep dive

The ixy_ixgbe.h header defines the specialized state and public interface for the Intel 82599 (ixgbe) family driver within the ixy framework. It serves as the bridge between the generic device abstraction and the specific hardware registers of the Intel 10GbE controllers.

The ixgbe_device structure is the primary handle for the driver. It follows a manual inheritance pattern by embedding struct ixy_device as its first member. This layout is a standard C idiom for polymorphism; since the address of the first member is the same as the address of the parent struct, a pointer to ixgbe_device can be safely cast to ixy_device for use in the generic framework. The struct also holds the addr pointer, which stores the virtual address of the MMIO (Memory Mapped I/O) space.

struct ixgbe_device {
	struct ixy_device ixy;
	uint8_t* addr;
	void* rx_queues;
	void* tx_queues;
};

The IXY_TO_IXGBE macro facilitates downcasting. It uses the container_of logic to retrieve the outer struct ixgbe_device pointer from the generic ixy_device pointer. This is essential for the driver's internal functions, which receive the generic type from the framework but must access ixgbe-specific fields like the MMIO base address or the private queue arrays.

Opaque Queue Pointers are utilized for rx_queues and tx_queues. By defining them as void* in the header, the driver hides the implementation details of the ixgbe-specific descriptor rings from the user. Only the implementation in ixgbe.c knows the internal structure of these queues, which involves circular buffers of hardware descriptors. This encapsulation prevents higher-level application code from accidentally tampering with the DMA-mapped memory regions used by the NIC hardware.

Public Entry Points cover the device lifecycle from initialization to statistics gathering. The ixgbe_init function is the constructor, responsible for mapping the PCI resources, resetting the controller, and setting up the initial descriptor rings. Management functions like ixgbe_set_promisc and ixgbe_set_mac_addr provide a clean API for modifying hardware filters. Instead of the user writing to specific RAL (Receive Address Low) or RAH (Receive Address High) registers, they call these high-level functions which encapsulate the register-level logic.

High-Performance Data Path functions are declared for batch processing. ixgbe_rx_batch and ixgbe_tx_batch are designed to handle multiple pkt_buf pointers in a single call. In userspace networking, batching is the primary mechanism to amortize the cost of MMIO "doorbell" writes—updating the hardware tail pointer—which is a relatively expensive operation over the PCIe bus.

uint32_t ixgbe_tx_batch(struct ixy_device* dev, uint16_t queue_id, 
                       struct pkt_buf* bufs[], uint32_t num_bufs);
uint32_t ixgbe_rx_batch(struct ixy_device* dev, uint16_t queue_id, 
                       struct pkt_buf* bufs[], uint32_t num_bufs);

Link and Statistics management is handled through ixgbe_get_link_speed and ixgbe_read_stats. These functions read the NIC's internal hardware counters. Note that ixgbe hardware counters are often "clear-on-read," so the driver must carefully manage these values to provide consistent device_stats to the application. While the header doesn't define queue limits, the Intel 82599 hardware typically supports up to 128 RX and TX queues, which the ixgbe_init function must validate against the requested parameters.

Interview angles

Polymorphism in C: Why is ixy_device the first member? This ensures that &ixgbe_dev->ixy == ixgbe_dev, allowing generic code to work with the specialized struct without complex offset calculations.
MMIO Abstraction: Why use uint8_t* addr instead of an integer? It allows for direct pointer arithmetic when accessing registers (e.g., addr + offset), which is how ixgbe.c implements the actual hardware register writes.
Batching Benefits: What is the performance impact of the num_bufs parameter? It reduces the frequency of PCIe doorbell writes and improves instruction cache locality by processing a burst of packets within a single function execution.

Going deeper

struct ixgbe_device {
	struct ixy_device ixy;
	uint8_t* addr;
	void* rx_queues;
	void* tx_queues;
};

Placing struct ixy_device as the first member allows safe pointer aliasing; a pointer to ixgbe_device can be safely cast to the generic ixy_device. The addr pointer represents the MMIO base address. Using void* for queues provides "pointer opacity," hiding hardware-specific ring descriptor structures from the public API to reduce header bloat and compile-time dependencies.

#define IXY_TO_IXGBE(ixy_device) container_of(ixy_device, struct ixgbe_device, ixy)

This macro uses offsetof to implement intrusive polymorphism. It allows the driver to retrieve its private hardware state from a generic interface pointer. Unlike a raw cast, container_of remains correct even if the ixy member's position changes, making it a staple for robust C system drivers.

Harder interview questions

Q: ixgbe_rx_batch accepts an array of pointers (bufs[]) rather than a flat buffer. Why? A: High-performance drivers use mempools. Pointers allow the driver to populate the array with pre-allocated, non-contiguous buffers from a pool, enabling out-of-order buffer recycling and avoiding massive memcpy operations.
Q: What happens if ixgbe_rx_batch is called by two threads on the same queue_id? A: There is no internal locking. Simultaneous access to the same descriptor ring would cause a race condition on the tail pointer register, leading to descriptor corruption or hardware "hangs" where the NIC stops processing the ring.

Gotchas

Memory Ordering: The header lacks explicit barriers. An implementation must use volatile and memory fences (like mfence or lfence) to ensure the CPU doesn't read packet data before the NIC has finished the DMA write to the descriptor.
MMIO Alignment: While addr is uint8_t*, register accesses (like setting the tail pointer) must be strictly 32-bit aligned. Unaligned or partial-word writes to ixgbe PCI BAR space can trigger bus errors or be ignored by the hardware.

From ixy to a production driver

Where the public API lives In ixy, ixgbe.h is almost the whole driver contract: allocate an ixy_device with ixgbe_init, use control calls, then move packets with ixgbe_rx_batch and ixgbe_tx_batch. Linux does not publish an ixgbe-specific application API. ixgbe.ko plugs into struct net_device, fills net_device_ops, registers ethtool_ops, and lets the kernel own naming, carrier state, feature flags, qdiscs, and lifecycle. DPDK is closer to ixy, but still hides the PMD behind struct rte_eth_dev, rte_eth_dev_ops, and the generic sequence rte_eth_dev_configure, queue setup, start, then bursts.

Queues and traffic steering The 82599 is a multiqueue NIC, not just two rings. Production ixgbe configures many RX/TX queues, MSI-X vectors, and RSS so flows spread across CPUs. Hardware state includes MRQC, the RETA redirection table, and the RSS key; DPDK exposes this through RSS configuration and rte_eth_dev_rss_reta_update. ixy accepts RX/TX queue counts, but the framework and examples are single-flow: enough for rings and DMA, not queue steering policy.

Interrupts, polling, and ownership ixy is pure poll-mode; interrupt_timeout is mostly a reminder that interrupts exist. Linux ixgbe uses MSI-X plus NAPI: interrupts signal work, NAPI polls rings under load, and interrupt moderation is programmed through EITR / coalescing. DPDK is poll-mode by default, but ethdev includes RX interrupt helpers such as rte_eth_dev_rx_intr_enable for power-aware loops. The interview point is that production drivers choose when to sleep, poll, and coalesce.

Concurrency contract ixy has no locks. If two threads drive the same queue, they race on descriptors and the ring tail. Linux must survive stack concurrency, so it uses netdev TX queue locking such as __netif_tx_lock, per-queue state, NAPI serialization, and reset coordination. DPDK makes the opposite production choice: burst functions are lock-free, but the documented contract is one lcore per queue unless the PMD advertises special support.

Features outside the fast path Real ixgbe exposes checksum offload, TCP segmentation offload (TSO/LSO), receive coalescing (RSC/LRO), VLAN offload, jumbo frames, flow control thresholds such as FCRTH/FCRTL, and management hooks. Linux advertises these via netdev->features and NETIF_F_*; DPDK reports capabilities through rte_eth_dev_info_get and enables offloads in rte_eth_conf. Linux ethtool_ops also covers link, rings, stats, register dumps, EEPROM, and flow control. ixy keeps only ixgbe_get_link_speed and ixgbe_read_stats, reading counters such as GPRC, GPTC, GORC, and GOTC.

What the omission teaches Production code must handle PCI hotplug, AER errors, runtime power management, resets, SR-IOV VFs, mailbox events, firmware/NVM quirks, retries, and bad hardware states. ixy drops that surface deliberately. Roughly 1000 lines explain DMA rings, MMIO registers, batching, and the poll-mode cost model before every concern a shipping driver owns.

Sources

Source

filesrc/driver/ixgbe.h

#ifndef IXY_IXGBE_H
#define IXY_IXGBE_H

#include <stdbool.h>
#include "stats.h"
#include "memory.h"

struct ixgbe_device {
	struct ixy_device ixy;
	uint8_t* addr;
	void* rx_queues;
	void* tx_queues;
};

#define IXY_TO_IXGBE(ixy_device) container_of(ixy_device, struct ixgbe_device, ixy)

struct ixy_device* ixgbe_init(const char* pci_addr, uint16_t rx_queues, uint16_t tx_queues, int interrupt_timeout);
uint32_t ixgbe_get_link_speed(const struct ixy_device* dev);
struct mac_address ixgbe_get_mac_addr(const struct ixy_device* dev);
void ixgbe_set_mac_addr(struct ixy_device* dev, struct mac_address mac);
void ixgbe_set_promisc(struct ixy_device* dev, bool enabled);
void ixgbe_read_stats(struct ixy_device* dev, struct device_stats* stats);
uint32_t ixgbe_tx_batch(struct ixy_device* dev, uint16_t queue_id, struct pkt_buf* bufs[], uint32_t num_bufs);
uint32_t ixgbe_rx_batch(struct ixy_device* dev, uint16_t queue_id, struct pkt_buf* bufs[], uint32_t num_bufs);

#endif //IXY_IXGBE_H