Phi-4 Bug Fixes

Phi-4, Microsoft’s new 14B model similar to OpenAI’s GPT-4o-mini, has been fine-tuned and debugged by Unsloth. The fixes include correcting the EOS token in the tokenizer and improving the padding token for better performance. Unsloth makes Phi-4 finetuning faster, more memory-efficient, and increases context length. The Phi-4 model has been converted to Llama’s architecture for enhanced accuracy and ease of use. Surprisingly, Reddit reports show that Unsloth’s fixes and Llama-fication of Phi-4 outperform Microsoft’s official model. Dynamic 4-bit quants have also been uploaded for faster downloading and increased accuracy. Overall, Unsloth offers significant improvements in Phi-4 performance.

https://unsloth.ai/blog/phi4

To top