Hello everyone! It’s been just over a month since we launched Helix v0.1, and today I’m excited to announce the availability of Helix v0.5. We’ve made significant improvements, including a new UI and enhanced text fine-tuning. We initially based our text fine-tuning on the Mistral-7B-Instruct language model from the LlamaIndex docs page. However, we soon realized that it had limitations, as it struggled with basic tasks. We had to implement multiple question-answer pairs to extract context from the documents effectively. By generating content-addressed hashes for each document, we taught the model about individual and grouped document IDs. We’re continuously working on improving prompts and systems using an “evals” framework and appreciate your feedback. Fine-tuning has proven advantageous in terms of information retention, latency, style copying, and understanding background knowledge. We’re excited to have an OpenAI compatible API and look forward to your engagement with the app. Stay tuned for updates on our roadmap and other interesting news.
https://helixml.substack.com/p/how-we-got-fine-tuning-mistral-7b