Ash Vardanian, the founder of Unum, discusses the untapped potential of modern CPUs in performing hyper-scalar operations through SIMD parallel processing. Highlighting the challenges in writing parallel operations, he shares insights from implementing SIMD kernels in the SimSIMD library. A surprising application of cosine similarity is showcased in various fields like farming robots and climate models. The article delves into optimizing cosine similarity code in C for different CPUs, emphasizing the use of bfloat16 numeric type and SIMD intrinsics for improved performance. The complexities of SIMD programming and techniques to address them are explored, setting the stage for a deep dive into AVX-512 implementation in Part 2.
https://www.modular.com/blog/understanding-simd-infinite-complexity-of-trivial-problems