The author dives into the use of the “vmsplice” system call to speed up data transfer through Linux pipes. They explore how using vmsplice can significantly boost performance, with some solutions reaching speeds up to 208.3 GiB/s. The author delves into the technical aspects of data transfer, comparing writing to cache versus writing to a pipe, and delves deep into the profiling of a program performing pipe writes. The surprising part is that even with optimizations, writing to a pipe is significantly slower compared to writing to memory directly due to various kernel operations and locking mechanisms.
https://qsantos.fr/2024/08/25/linux-pipes-are-slow/