Torchchat allows you to seamlessly run large language models (LLMs) using Python in various environments, including desktop, server, iOS, and Android. You can interact with popular LLMs such as Llama 3 and Mistral through command-line interfaces or use PyTorch-native execution for optimal performance. The tool supports multiple hardware and operating systems, including Linux, macOS, Android, and iOS. Unique features include the ability to chat with models interactively, generate text based on prompts, and deploy models for mobile execution. While still a work in progress, torchchat offers a wide range of functionalities for model deployment and evaluation.
https://github.com/pytorch/torchchat