Mozilla is committed to developing trustworthy AI systems. They believe in the power of open source technology to ensure transparency and prevent bias. The team at Mozilla recently undertook a project to build an internal chatbot prototype that runs entirely on Mozilla’s cloud infrastructure and uses open source language models. They faced challenges in hosting, runtime environments, model selection, and prompt engineering. They eventually settled on using llama.cpp, an open source project that allows them to run LLaMa architecture-based models. They also used embedding and vector search techniques to incorporate internal company knowledge into the chatbot. The process of prompt engineering was crucial in ensuring the chatbot’s behavior aligns with Mozilla’s principles.
https://hacks.mozilla.org/2023/07/so-you-want-to-build-your-own-open-source-chatbot/