We have conducted a study where we applied Large Language Models to code optimization. Specifically, we trained a 7B-parameter transformer model to optimize LLVM assembly for code size. The model takes unoptimized assembly as input and generates a list of compiler options that would best optimize the program. To enhance the model’s performance and understanding, we incorporated additional learning tasks that required it to predict instruction counts and generate optimized code. Our evaluation on a comprehensive set of test programs revealed that our approach achieved a 3.0% improvement in reducing instruction counts compared to the compiler. Surprisingly, the model demonstrated strong code reasoning abilities, producing compilable code 91% of the time and perfectly emulating compiler output 70% of the time.
https://arxiv.org/abs/2309.07062