NVIDIA has developed a fork of its driver with P2P support for 4090 GPUs. By installing with ./install.sh, users can enable large BAR support and turn off IOMMU. The driver utilizes PCIe instead of MAILBOXP2P due to hardware limitations on 4090s, enhancing memory transfer between GPUs. Despite initial challenges like crashes and errors, through direct method calls and adjustments, P2P functionality can be achieved. The driver’s stable performance, evidenced by successful data transfers and test results, showcases its capabilities. With potential for further improvements and upstream integration, NVIDIA’s innovative approach enhances GPU communication and paves the way for more efficient processing.
https://github.com/tinygrad/open-gpu-kernel-modules