When it comes to GenAI, the association with GPUs usually leads the conversation towards Nvidia products. Nvidia has created a strong software “moat” around its hardware by providing software tools like CUDA and optimized libraries like cuDNN. However, this has created difficulties for companies and users who want to enter the HPC and GenAI market with alternate hardware. Many foundational models for GenAI are open source and require a large number of resources to create. Fine-tuning and inference tasks in GenAI require accelerated computing, preferably with GPUs. While Nvidia remains dominant, other vendors like AMD are making strides with tools like the HIP CUDA conversion tool and the PyTorch framework. The upcoming Instinct MI300A processor from AMD is expected to compete with Nvidia’s Grace-Hopper superchip in both HPC and GenAI. The battle for the GenAI market will be won by performance, portability, and availability.
https://www.hpcwire.com/2023/10/05/how-amd-may-get-across-the-cuda-moat/