io_uring Zero-Copy Receive Trades Copies for Buffer Ownership

You replace an epoll + recvmmsg loop with io_uring multishot receive. Syscalls drop, the CPU flame graph looks cleaner, and throughput barely moves. That result only feels strange if zero-copy receive made you picture the send path in reverse.

The missing throughput is the tell: on receive, the expensive boundary often is not the syscall you removed. It is the ownership hop you have not changed yet.

In episode 1, we drew the ownership ledger. In episode 2, TLS inserted a new representation. Receive-side io_uring changes a different boundary again: the kernel already owns the arriving bytes first.

TL;DR: Stable ordinary io_uring receive mostly removes submission, wakeup, and buffer-selection overhead. io_uring zero-copy receive (ZCRX) first shipped in Linux 6.15. This post uses Linux 7.1 and liburing 2.15 as the released baseline: mixed-size completion queue entries (CQEs), receive-area chunk hints, and areas backed by DMA-BUF, Linux's cross-device shared-buffer interface, are no longer next-tree-only. Event-backed copy-fallback statistics still are.

What you'll learn:

• Why ordinary multishot/provided-buffer receive is useful even when it still copies
• What stable ZCRX requires in released kernels, and what is still current-upstream-only
• How the buffer-state machine changes between copied-path receive and ZCRX
• How to benchmark receive-side copy avoidance without lying to yourself

On receive, the copy disappears only when buffer ownership can move safely. Stable io_uring makes receive cheaper first; true zero-copy receive makes ownership harder.

Receive-side zero-copy starts with borrowed memory, not a faster recv()

Send-side copy avoidance preserves application-owned bytes. Receive-side copy avoidance has to loan kernel-fed RX memory forward.

That asymmetry is the whole story. On send, the application already owns the payload and asks the kernel not to copy it into private socket storage. On receive, the network interface card (NIC) and kernel receive path own the first useful representation. The packet lands through direct memory access (DMA) into receive memory, gets processed by New API (NAPI) polling, and only then becomes something your socket can expose to user space per Linux's NAPI documentation and the page_pool documentation.

That is why ordinary receive and receive-side zero-copy need different mental models. The copied path is simple:

NIC DMA -> RX memory [kernel] -> socket receive path [kernel] -> user buffer

A true receive-zero-copy path needs a different promise:

NIC DMA -> RX-backed memory [kernel-owned first] -> loaned view in user space
       -> explicit recycle back to the receive path

Linux already shows this pattern in another subsystem. AF_XDP, the XDP socket interface, documents that RX zero-copy depends on driver support, queue binding, UMEM ownership, and explicit fill/completion rings rather than on a magical faster recv() call in the kernel AF_XDP docs. io_uring ZCRX follows the same physics even though the socket-facing API looks friendlier.

For your service, this means: receive-side zero-copy is usually a pool and locality decision before it is an API decision.

Ordinary multishot receive is stable today, but it still copies into your buffers

Provided buffers, buffer rings, multishot receive, and receive bundles are real wins. They are not the same thing as stable ZCRX.

The stable ordinary path is well documented in liburing 2.15. Provided buffers let you pre-register candidate receive buffers, a buffer ring lets the kernel pick one cheaply, and multishot receive keeps one receive request armed across several completions instead of resubmitting after every packet or read per io_uring_provided_buffers(7), io_uring_setup_buf_ring(3), and io_uring_prep_recv(3). Receive bundles are also stable; liburing documents IORING_RECVSEND_BUNDLE as part of the provided-buffer receive path, not as ZCRX.

That distinction matters because the ordinary path still ends with the kernel copying payload bytes into the selected userspace buffer. The completion tells you which buffer won. It does not prove that the copy disappeared. IOU_PBUF_RING_INC, documented as available since Linux 6.12, extends the provided-buffer model with incremental consumption, but it stays in that family per liburing 2.15.

For most applications, the ordinary io_uring path is the right first move. It lowers submission churn and wakeups without demanding special NIC features or a new refill protocol. You should only pay the ZCRX complexity tax after the copied path is already clean and measured.

Here is the smallest realistic stable baseline to make that contract visible:

enum { BGID = 7, BUF_COUNT = 1024, BUF_LEN = 2048 };
 
struct io_uring_buf_ring *br =
    io_uring_setup_buf_ring(&ring, BUF_COUNT, BGID, 0, &ret);      // ①
unsigned mask = io_uring_buf_ring_mask(BUF_COUNT);
 
for (unsigned i = 0; i < BUF_COUNT; i++) {
  io_uring_buf_ring_add(br, bufs[i], BUF_LEN, i, mask, i);         // ②
}
io_uring_buf_ring_advance(br, BUF_COUNT);                          // ③
 
struct io_uring_sqe *sqe = io_uring_get_sqe(&ring);
io_uring_prep_recv_multishot(sqe, sockfd, NULL, 0, 0);            // ④
sqe->flags |= IOSQE_BUFFER_SELECT;
sqe->buf_group = BGID;                                             // ⑤

① The ring holds candidate receive buffers in user space; this is the stable provided-buffer path, not ZCRX.

② Each slot gets a stable buffer ID. That ID is how you recover ownership on completion.

③ Publishing the tail makes the buffers visible to the kernel in one step.

④ Multishot receive keeps one request armed across several completions.

⑤ IOSQE_BUFFER_SELECT tells the kernel to choose from your buffer group instead of a single explicit pointer.

When completions arrive, you still have to respect lifetime:

if (cqe->res < 0 || !(cqe->flags & IORING_CQE_F_BUFFER))
  handle_recv_completion_error(cqe);
 
unsigned bid = cqe->flags >> IORING_CQE_BUFFER_SHIFT;              // ①
char *buf = bufs[bid];
 
parse(buf, cqe->res);                                              // ②
 
if (!(cqe->flags & IORING_CQE_F_MORE)) {                           // ③
  arm_another_multishot_recv(&ring, sockfd, BGID);
}
 
io_uring_buf_ring_add(br, buf, BUF_LEN, bid, mask, 0);            // ④
io_uring_buf_ring_advance(br, 1);                                  // ⑤

① IORING_CQE_F_BUFFER gives you the winning buffer ID. It does not say anything about whether the payload was copied or loaned.

② The buffer is yours to parse now, so hold time starts here.

③ IORING_CQE_F_MORE means the multishot receive is still armed. It is not a buffer reuse signal.

④ Re-adding the buffer before parsing finishes is a correctness bug even on the copied path.

⑤ Recycling one slot at a time makes hold-time accounting easy to instrument.

The practical decision rule is simple. If your current pain is syscalls, wakeups, and resubmission churn, ship this path first. If memory bandwidth is still your hot boundary after that, ordinary multishot receive has done its job: it gave you a clean baseline for asking whether ZCRX is worth the rest.

Stable ZCRX is a separate contract, not a flag on the old path

Released Linux docs treat ZCRX as its own networking feature with its own ring setup, memory registration, and refill protocol.

Before looking at setup code, pin the version line down. ZCRX first appears in released Linux 6.15 documentation. This post uses Linux 7.1 and liburing 2.15 as its stable baseline. If you are on Linux 6.14 or earlier, you still have the ordinary receive features from the previous section, but not the released ZCRX contract.

The released API is built around a registered interface queue, a userspace payload area, and a dedicated refill ring, the ring that returns consumed payload slots to the kernel. liburing 2.15 mirrors that split through io_uring_register_ifq(3), io_uring_register_region(3), io_uring_register_query(3), and io_uring_register_zcrx_ctrl(3).

Linux 7.1 requires IORING_SETUP_SINGLE_ISSUER, IORING_SETUP_DEFER_TASKRUN, and either IORING_SETUP_CQE32 or IORING_SETUP_CQE_MIXED for ZCRX setup in the released docs and liburing 2.15's IFQ manual. The NIC still needs header/data split, flow steering, and receive-side scaling (RSS) so the kernel TCP path can keep headers while payload DMA lands in the userspace-backed area. Current driver guidance goes deeper: drivers need the page_pool and network memory (netmem) handoff model in Linux 7.1's netmem documentation.

That is already enough to separate ZCRX from the ordinary path:

• ordinary multishot receive: provided buffers, buffer IDs, copied payloads;
• stable ZCRX: registered interface queue, registered payload region, dedicated refill ring, 32-byte or mixed-size CQEs, and RX-backed payload offsets;
• Linux 7.1 additions already in released UAPI: rx_buf_len chunk hints, DMA-BUF-backed payload areas, and import or device-less registration flags that applications must discover before use. ZCRX_REG_NODEV is explicitly a copied, device-less mode, not another zero-copy path in the Linux 7.1 header.

This is the stable shape, trimmed to the lifetime edges and written as pseudocode after the released docs plus liburing 2.15's examples/zcrx.c:

struct io_uring_params p = {
  .flags = IORING_SETUP_SINGLE_ISSUER |
           IORING_SETUP_DEFER_TASKRUN |
           IORING_SETUP_CQE_MIXED,                                // ①
};
io_uring_queue_init_params(DEPTH, &ring, &p);
 
struct io_uring_zcrx_area_reg area = {
  .addr = (uintptr_t)area_ptr,
  .len = area_len,
};
struct io_uring_region_desc refill = make_refill_region(rq, rq_len);
 
struct io_uring_zcrx_ifq_reg ifq = {
  .if_idx      = if_nametoindex("ens1f0np0"),
  .if_rxq      = 3,
  .rq_entries  = rq_entries,
  .area_ptr    = (uintptr_t)&area,
  .region_ptr  = (uintptr_t)&refill,
  .rx_buf_len  = 0,                                               // ②
};
io_uring_register_ifq(&ring, &ifq);                              // ③
 
sqe = io_uring_get_sqe(&ring);
sqe->opcode = IORING_OP_RECV_ZC;
sqe->fd = sockfd;
sqe->ioprio = IORING_RECV_MULTISHOT;
sqe->zcrx_ifq_idx = ifq.zcrx_id;                                 // ④
 
/* later, on CQE */
struct io_uring_zcrx_cqe *zcqe = (void *)(cqe + 1);
payload_ptr = area_ptr + (zcqe->off & ~IORING_ZCRX_AREA_MASK);
process(payload_ptr, cqe->res);
zcrx_refill(rq, zcqe->off, cqe->res, area.rq_area_token);        // ⑤

① ZCRX is not ordinary receive with one more flag. Linux 7.1 permits mixed CQEs; IORING_SETUP_CQE32 remains valid when every CQE is 32 bytes.

② Stable ZCRX registers a ZCRX payload area and a separate refill-ring region. A zero chunk hint asks for the default page-size behavior.

③ io_uring_register_ifq() binds the setup to a specific interface queue and requires CAP_NET_ADMIN per liburing 2.15.

④ The receive request names the registered ZCRX interface queue, not a provided-buffer group.

⑤ The CQE offset and refill write are the ownership handoff and return path. The area token comes from registration, not from the CQE. Until you refill, the slot is not reusable.

One naming trap is worth making explicit. Stable io_uring_register_query(3) covers IO_URING_QUERY_ZCRX. Linux 7.1's released query structure reports registration flags, area flags, control operations, and supported features in the versioned header.

Current upstream goes farther with IO_URING_QUERY_ZCRX_EVENT, ZCRX_EVENT_COPY, cumulative copy counters, and ZCRX_CTRL_ARM_EVENT in current query.h and zcrx.h. Those event-backed fallback signals are not in Linux 7.1's released UAPI, so keep them version-gated.

For your environment, this means: if you cannot name the kernel version, driver, queue, and refill path, you do not yet have a ZCRX design. You only have optimism.

The buffer-state machine is the feature

ZCRX earns its place only when you can reason about every slot's owner from free to refill.

The ordinary copied path and stable ZCRX differ less in opcodes than in state transitions. A provided buffer starts free in your ring, gets selected by the kernel, receives a copied payload, becomes application-owned at CQE time, and goes back to your ring after parsing. The kernel never needs that exact userspace slot again once it finishes the copy.

ZCRX is stricter because the kernel wants the same RX-backed capacity back. The slot is free in the refill ring, becomes kernel-owned for receive, becomes user-visible through a CQE offset into the payload area, stays application-owned while you parse or retain it, and only becomes reusable when you explicitly return it to the refill ring. The state machine is the optimization.

ordinary provided-buffer receive
free in buf ring
  -> kernel selects user buffer
  -> payload copied into that buffer
  -> CQE returns buffer ID
  -> application parses
  -> application re-adds buffer to buf ring
 
stable ZCRX
free in refill ring
  -> kernel/NIC RX path owns payload slot
  -> CQE returns payload offset in registered area
  -> application parses or retains
  -> application pushes offset back to refill ring
  -> slot becomes eligible for RX again

Two details decide whether this stays correct under load. First, the multishot receive can stay armed across many completions, so IORING_CQE_F_MORE still describes request lifetime rather than buffer lifetime, just as it does on the copied path per liburing 2.15's example. Second, backpressure now shows up as refill starvation rather than only as socket backlog growth. A slow parser is no longer just slow. It can directly reduce RX capacity.

This is where the hardware requirements stop sounding decorative. Linux's ZCRX docs require header/data split, steering, and RSS because locality is part of correctness here, not just speed in Linux 6.15 and Linux 6.16. The generic networking docs explain why: queue placement, CPU affinity, and receive-side steering decide which core handles packets and where memory traffic lands per the scaling docs and the NAPI docs. If payload DMA lands on one NUMA node and your parser runs on another, you can erase the saved copy and keep the lifetime pain.

What could go wrong

• You benchmark the copied path and call it ZCRX. Ordinary multishot receive, provided buffers, incremental buffer rings, and receive bundles are documented separately from ZCRX and still belong to the copied receive family per liburing 2.15 io_uring_prep_recv(3), io_uring_provided_buffers(7), and the Linux 7.1 ZCRX docs.
• Your NIC or driver misses the prerequisites. Released ZCRX docs require header/data split, flow steering, and RSS, while driver guidance adds page_pool and netmem expectations per Linux 7.1 and the versioned netmem docs.
• Refill starvation becomes a concrete failure mode, not a theory. liburing 2.15's reference zcrx.c handles -ENOSPC by resubmitting the receive and logs the case where a full refill queue drops the returned buffer in the example. If you do not count those paths, you do not know whether the zero-copy path held up under load.
• Fleet policy blocks the path before your code gets interesting. Linux documents io_uring_disabled as an attack-surface control in the 7.1 sysctl guide. ZCRX registration also needs CAP_NET_ADMIN, so "the benchmark worked" is not the same as "the fleet can deploy it" per liburing 2.15.
• You assume every released option works on every device. Linux 7.1 exposes rx_buf_len, DMA-BUF areas, and import flags, but chunk sizing still depends on kernel and hardware support. Query capabilities, retry chunk registration with rx_buf_len = 0 when needed, and keep a copied path per the released docs and released UAPI.
• You collapse stable query support and current-upstream event extensions into one bucket. Stable IO_URING_QUERY_ZCRX and current-upstream IO_URING_QUERY_ZCRX_EVENT are not the same release contract per liburing 2.15, Linux 7.1's query header, and current query.h.

A benchmark only counts if it proves which path ran

Do not headline throughput first. Headline the path, the ownership contract, and the fallback story.

For ordinary multishot receive, proving the path is easy: show buffer-ring setup, CQE buffer IDs, and recycle lag. For ZCRX, the minimum proof is stronger: successful interface-queue registration, required ring flags, 32-byte ZCRX trailer data, offsets that point into the registered payload area, and a refill stream returning those offsets to the kernel per Linux 7.1 and liburing 2.15's example. If you use current-upstream event-backed copy counters, label them as version-gated extras rather than part of the Linux 7.1 contract.

My minimum matrix for this topic is four-way:

1. epoll + recvmmsg
2. io_uring multishot + provided buffers
3. io_uring receive bundle
4. stable ZCRX

Keep the same NIC, queue, MTU, coalescing, parse logic, and CPU affinity across all four paths. Then run at least three parser modes: discard, header-only, and touch-every-byte. That split matters because header-light workloads are where receive-side copy avoidance has the clearest chance to win; if your parser walks every byte anyway, memory-touch cost can dominate the copy you removed. The kernel networking docs on RSS, queue scaling, and NAPI are the right checklist for pinning IRQ, softirq, and application work to the same locality domain per the scaling docs and the NAPI docs.

If you cite the upstream prototype, keep the caveat glued to the number. LWN's coverage of the RFC reports roughly 50% lower overall system memory bandwidth for ZCRX than the non-zero-copy path on Broadcom BCM57504 plus Intel Xeon Platinum 8321HC, measured with perf counters, and the authors explicitly disabled Intel Data Direct I/O (DDIO). That is good evidence that ZCRX can relieve bandwidth pressure. It is not a universal "ZCRX halves memory traffic" rule for hardware still running with DDIO on or for a different cache topology.

Collect:

• throughput and tail latency;
• cycles per byte, LLC misses, and memory-bandwidth counters with perf stat;
• CPU flame graphs or samples with perf record;
• CQEs per second, final CQEs without F_MORE, and buffer hold-time histograms;
• refill starvation, -ENOSPC, and recycle lag on the ZCRX path.

The point of that list is not ceremony. It is to stop you from shipping a graph that says "zero-copy receive was faster" when the only confirmed change was that your receive loop submitted less work.

Key takeaways

• Stable ordinary io_uring receive is already useful because it removes control-plane overhead, even when the kernel still copies payloads into your buffers.
• ZCRX first shipped in Linux 6.15; the Linux 7.1 contract adds released capabilities without changing the core interface-queue, payload-area, and refill-ring lifetime model.
• The hard part of receive-side zero-copy is not the opcode. It is buffer ownership, queue locality, and safe recycle under load.
• A receive benchmark is not a zero-copy benchmark unless it proves which path ran, which version contract you used, and how buffers returned.

What's next: Episode 4 moves the boundary again and compares AF_XDP with DPDK, where zero-copy wins by giving user space much more of the NIC contract.

Sources and References

Linux ZCRX and io_uring API

• Original released contract: Linux 6.15 introduced ZCRX payload areas and refill rings - Linux 6.15 ZCRX docs
• Current released baseline: Linux 7.1 documents mixed CQEs and area chunking - Linux 7.1 ZCRX docs
• Released UAPI capabilities: Linux 7.1 exposes DMA-BUF areas, import and device-less registration flags, and rx_buf_len - Linux 7.1 zcrx.h
• Interface-queue registration and privilege requirement: io_uring_register_ifq() documents IFQ registration and CAP_NET_ADMIN - liburing 2.15 io_uring_register_ifq(3)
• Memory-region registration: userspace refill regions use region descriptors - liburing 2.15 io_uring_register_region(3)
• Stable ZCRX query surface: io_uring_register_query() documents IO_URING_QUERY_ZCRX - liburing 2.15 io_uring_register_query(3)
• Current-upstream event extensions: current headers add IO_URING_QUERY_ZCRX_EVENT, fallback statistics, and event arming - current query.h and zcrx.h
• Stable ZCRX control operations: flush and export controls are documented through the ZCRX control man page - liburing 2.15 io_uring_register_zcrx_ctrl(3)
• Ordinary receive bundles are not ZCRX: IORING_RECVSEND_BUNDLE belongs to the provided-buffer receive path - liburing 2.15 io_uring_prep_recv(3)
• Buffer-ring setup and incremental consumption: liburing 2.15 io_uring_setup_buf_ring(3) and io_uring_provided_buffers(7)
• Stable example code: refill handling, -ENOSPC recovery, DMA-BUF setup, and full-refill-queue behavior - liburing 2.15 examples/zcrx.c

Receive-path prerequisites and memory ownership

• RX memory recycling model: Linux 7.1 page_pool docs
• Driver-side memory handoff expectations: Linux 7.1 netmem.rst
• Queue steering and locality: Linux 7.1 scaling docs
• Polling and receive execution context: Linux 7.1 NAPI docs
• A stable comparison point for RX zero-copy: Linux 7.1 AF_XDP docs

Security and benchmark tooling

• Operational security control: io_uring_disabled can reduce kernel attack surface - Linux 7.1 kernel sysctl docs
• Counter collection for benchmark method: Linux 7.1 perf stat docs
• Sampling for benchmark method: Linux 7.1 perf record docs

Prototype data and design discussion

• Prototype benchmark caveat: early ZCRX reporting described roughly 50% lower overall system memory bandwidth on Broadcom BCM57504 plus Intel Xeon Platinum 8321HC, with Intel DDIO disabled - LWN coverage of the ZCRX RFC