Skip Navigation

How fast are Linux pipes anyway?

mazzo.li How fast are Linux pipes anyway?

Pipes are ubiquitous in Unix --- but how fast can they go on Linux? In this post we'll iteratively improve a simple pipe-writing benchmark from 3.5GiB/s to 65GiB/s, guided by Linux `perf`.

How fast are Linux pipes anyway?
2
TechNews @radiation.party irradiated @radiation.party
BOT
[HN] How fast are Linux pipes anyway?
2 comments
  • Quality post, thanks for sharing!

  • We will proceed as follows:

    1. A first slow version of our pipe test bench;
    2. How pipes are implemented internally, and why writing and reading from them is slow;
    3. How the vmsplice and splice syscalls let us get around some (but not all!) of the slowness;
    4. A description of Linux paging, leading up to a faster version using huge pages;
    5. The final optimization, replacing polling with busy looping;
    6. Some closing thoughts.