Slow-Path Splitting

Ruminations of a Programmer，编程、数学和技术深度探讨关注

Part 4 of Low-Level Systems Design in Rust - a series on writing high-throughput, low-latency systems code, using a single-producer / single-consumer (SPSC) ring buffer as the running example.Part 1 decided where the shared cursors of a concurrent structure live in memory.Part 2 covered how two cores read and write them correctly and at minimum cost.Part 3 made those reads and writes rare.

This post is about the shape of the operation itself - moving data without copying it, and structuring each call so the common case is a branch the CPU can predict and pipeline. The running example is still a single-producer / single-consumer (SPSC) ring buffer, but neither technique is specific to ring buffers.

There are code snippets as part of the post. If you want to take a deeper dive into the ring buffer project, take a look at the code, tests and Quint specifications at the repo.

Two taxes that have nothing to do with atomics

Parts 1 through 3 were all about the cost of coordination - cache lines, memory ordering, and how often cores have to talk. But a naive hot path pays two more taxes that no amount of clever atomics will fix:

Copying. In the obvious queue API the producer builds an item somewhere (on its stack), then push(item) copies it into a ring slot - that's the copy-in. Later the consumer calls pop(), which returns the item by value, meaning it gets copied back out of the slot into the consumer's local variable - that's the copy-out. For anything larger than a machine word that's two memcpy s per item the design can avoid. This post presents the reserve/commit protocol that builds the producer side, which eliminates the copy-in. The consumer's copy-out is really the mirror-image protocol - borrow the initialized slots in place, process them, then release.

Mixing the rare case with the common one. If every call runs the expensive path (synchronize, allocate, syscall) inline wi…

添加评论

点踩

分享查看原文

参与讨论