Qualcomm’s NPU on the Microsoft Surface Tablet is tested in this benchmark, aiming to showcase its performance as an AI PC. Lack of code examples and benchmarks for external developers prompted the creation of this project. However, the results fall below expectations, highlighting the need for improvements in latency. The installation process involves Python, Cmake, Visual Studio, and Pip Packages to run the benchmark. The benchmark measures the computational throughput of the CPU and NPU using large matrix multiplications resembling real-world models. Surprisingly, the NPU results are slower than the CPU, and only achieve 1.3% of the marketed performance, raising questions about optimizing software and improving results.
https://github.com/usefulsensors/qc_npu_benchmark