⏱️ Timestamping & capture
Solarflare's finance/telecom edge: knowing exactly when a packet crossed the wire, and capturing every one at line rate.
7.1 Wire-time timestamps & ordering
Why a late timestamp is a useless timestamp. Suppose a frame lands on the wire at time T. By the time software learns the frame exists, it has crossed the PCS and MAC layers, sat in an elastic buffer that absorbs the small frequency difference between the recovered receive clock and the local clock, crossed one or more clock-domain synchronisers, been DMA'd into host memory, possibly waited on an interrupt or a polling cycle, and finally surfaced through a descriptor ring. Every one of those stages adds latency — and worse, variable latency. An elastic buffer's fill level depends on traffic, interrupt coalescing depends on load, polling cadence depends on what the CPU was doing. So a timestamp taken when software notices the packet is really T plus an unknown, jittering offset. For finance (proving which order reached the matching engine first) and telecom (IEEE 1588 / PTP distributing a clock to within nanoseconds), that jitter is the whole error budget. The fix is to take the timestamp where the jitter hasn't accumulated yet.
`US10742782B2` — timestamp at the physical datapath, compensated to a bit. This patent (Taylor, Pope, Riddoch, Roberts; Solarflare, now Xilinx) is exactly about generating timing information around the Ethernet receive/transmit path and dealing with the PCS/MAC clock-domain reality. The naive approach — latch a counter when "the packet arrives" — is ambiguous, because the packet doesn't arrive at an instant; it streams in over many clock cycles across two clock domains. The patent describes capturing timing near the physical layer and then compensating so the reported timestamp corresponds to a specific, well-defined part of the frame — a particular bit or byte, such as the start of the frame. That definition matters: only if both the sender and receiver agree the timestamp means "the moment bit N crossed the boundary" can two timestamps be compared or a PTP offset be computed. The compensation accounts for the known, fixed delays of the PCS/MAC pipeline and the elastic buffer, converting a measurement taken in the device's clock domain into a wire-referenced instant. Taking it before DMA, interrupts, and polling means the variable host-side latency never enters the number at all.
`US9778963B2` / `US10409655B2` — ordering, not just readiness. A wire-accurate timestamp is wasted if the application then processes packets in the wrong order. A classic epoll/select loop tells you which sockets are ready, but not the order in which their data arrived on the wire — and when you poll many sockets or queues, the act of scanning them imposes its own arbitrary order. For a trading system watching dozens of market-data feeds, "socket 7 became readable before socket 3" is meaningless; what matters is which byte hit the NIC first. These two patents (Pope, Riddoch, Mansley, James) describe an event-notification mechanism that tells the application not only which sockets are ready but the order in which the ready events should be processed, tying that order to per-segment data, byte counts, and hardware timestamps taken by the NIC. The readiness primitive is upgraded from a set into a sequence, with the sequence anchored to wire time rather than to whatever order the poller happened to walk its file descriptors.
Why this matters for low-level NIC work. The discipline here is to attach truth to the packet at the earliest physical point and then never lose it. The timestamp is born at the PCS/MAC boundary, compensated to a defined bit of the frame, and carried — through RX descriptors, event queues, and the user-space stack — alongside an ordering guarantee, so that everything downstream reflects what actually happened on the wire. An interviewer will probe both halves: where exactly in the RX/TX pipeline the counter is sampled (and how elastic buffers and clock-domain crossings are compensated), and how that wire-time metadata survives the trip into user space without the polling loop silently reordering it.
Sources
7.2 Line-rate capture
Capture is not a copy — it is preserving order, gaps, and timing under pressure. It is tempting to think packet capture is just "save everything," a throughput problem solved by a fast disk. It is not. The hard part is that a capture is only useful if it faithfully reproduces what happened on the wire: which packet came first, how long the gap was between two of them, and the exact wire time of each. The moment you let the operating system handle a packet before you capture it, you have already lost the timing; the moment your consumer (an analyser or a disk) falls behind during a burst, you risk either dropping packets or silently reordering the stream. The Solarflare capture patents — the lineage behind SolarCapture — treat capture as a first-class NIC datapath function precisely so that none of this metadata is lost.
`US9300599B2` — capture inside the NIC, on the live path. This patent (Pope, Riddoch, Porter) describes a packet-capture unit inside the network interface device. It inspects packets and can duplicate a flow into a separate packet-capture data stream while the original traffic continues, unmodified, on its normal path — so capturing does not perturb the production datapath. The duplicate is DMA'd into a dedicated host capture buffer, and the patent covers replaying captured traffic using its timestamp information to reproduce the relative receive timing. Capture happens at the wire, with wire-time attached, before the OS or application has a chance to distort timing or order.
`US10733167B2` — feeding an analyser, and surviving when it can't keep up. Capturing at line rate is easy compared with consuming at line rate: an analytics device frequently cannot process every burst in real time. This patent (Pope, Riddoch, Knight) describes converting captured data into a timestamped output format and feeding an analyser through a buffering / packet-injection path, with the key trick of switching between live data and stored data when the consumer falls behind. Timestamps keep delivery ordered across that live/stored boundary, so the analyser sees one coherent, correctly ordered stream even though some of it came straight off the wire and some was pulled back out of storage after a burst.
`US10691661B2` — storage that prioritises the writer. The companion storage patent (Pope, Riddoch, Knight) covers the other end: writing captured packet data and indexes to a block store, deliberately prioritising capture writes over reads. The asymmetry is the whole design: packets arrive on a deadline you do not control, so the writer must win contention against query/read traffic. Analytics and query consumers are allowed to catch up later from stored data, using timestamp metadata to process stored and live data in the correct order. Indexing alongside the raw bytes is what later makes "show me everything on this flow between these two times" answerable without rescanning the whole capture.
`US10079919B2` — speculative receive. The last piece attacks the latency cost of correctness. Normally an application sees received data only after protocol processing has fully completed (checksums verified, ordering confirmed). This patent exposes received data to the consumer before that processing finishes, then commits on success — and discards or invalidates if a check later fails. For capture and low-latency receive alike, that gets data into the consumer's hands at the earliest possible moment, paying the rollback cost only on the rare failure rather than the per-packet validation cost on every success.
Why this matters for low-level NIC work. The throughline is that line-rate capture is a backpressure and metadata-preservation problem, not a bandwidth one. Capture on the live path with wire-time attached, buffer and switch between live and stored when the consumer stalls, let the writer beat the reader to disk, index as you go, and speculatively expose data while keeping a rollback path for correctness. An interviewer will ask how the system decides between live delivery, in-memory buffering, disk-backed catch-up, and replay when the analyser cannot sustain the input rate — and the answer is always: never drop the order, the gaps, or the timestamps, because those are the only things that make the capture mean anything later.
Sources