Candle is a minimalist ML framework for Rust that aims to be easy to use and high-performing, even with GPU support. The website features online demos, such as Whisper for speech recognition and Llama-v2 for a general LLM. The framework offers simple syntax reminiscent of PyTorch and supports CPU and CUDA backends. It also enables serverless deployments, lightweight deployments with WebAssembly (WASM) support, and distributed computing using NCCL. Candle provides pre-built models like Llama, Whisper, Falcon, and StarCoder, and allows users to embed custom ops/kernels. The FAQ section addresses questions about Candle’s motivation and comparisons with other ML frameworks.
https://github.com/huggingface/candle