Answer.AI is introducing an open-source system allowing efficient training of a 70 billion parameter language model on a regular desktop computer with gaming GPUs. Collaborating with industry experts from U Washington and Hugging Face, this project aims to democratize access to advanced AI models. By combining QLoRA and FSDP, they’ve made it possible to train these large models without the need for expensive hardware. The unique approach tackles challenges faced by academia, big tech, and startups, making AI development more accessible. Despite initial obstacles, the team successfully fine-tuned a 70b model on dual 3090 gaming GPUs, showcasing a groundbreaking achievement in AI research.
https://www.answer.ai/posts/2024-03-06-fsdp-qlora.html