Ollama for Linux is an exciting new development that brings GPU acceleration to the forefront. This means that Nvidia GPUs can now harness this power straight out-of-the-box, resulting in enhanced performance. Additionally, Ollama can be deployed on cloud servers with multiple GPUs, ensuring that scalability is not an issue. The integration with WSL 2 brings GPU support to this platform as well. Surprisingly, Ollama is adept at loading a high number of GPU layers without crashing, thereby maximizing performance. From smaller hobby gaming GPUs to powerful workstation graphics cards like the H100, Ollama supports a wide range of hardware. To install, simply run the provided command or follow the manual install steps. Notable updates include automatic offloading for optimal performance and the addition of an exciting community project.
https://github.com/jmorganca/ollama/releases/tag/v0.1.0