Rust std fs slower than Python? No, it’s hardware

In this web content, the author shares their experience and findings regarding the performance of OpenDAL’s Python binding compared to Python itself. The initial benchmark results show that the Python binding is slower than Python, which surprises and embarrasses the author. They propose hypotheses and refactor the code to investigate the reasons behind this. Further benchmark tests reveal that both Rust and Python implementations of OpenDAL are slower than Python alone. The author then explores the use of mmap and memory allocators in Python and Rust, finding that switching to the jemalloc memory allocator significantly improves Rust’s performance. Additionally, the author discovers that Rust is only slower on their particular machine, suggesting a potential hardware-related issue.

https://xuanwo.io/2023/04-rust-std-fs-slower-than-python/

To top