In this report, the authors explore the possibility of programming languages boosting each other during the instruction fine-tuning phase of code large language models. The study focuses on 8 popular programming languages including Python, JavaScript, TypeScript, C, C++, Java, Go, and HTML. The results of extensive experiments conducted on StarCoder show that programming languages can significantly improve each other. For instance, CodeM-Python 15B, trained on Python, enhances Java performance by 17.95% pass@1 on HumanEval-X. Interestingly, CodeM-HTML 7B, trained on the HTML corpus, improves Java by 15.24% pass@1. The training data used in the study is available at the provided URL.
https://arxiv.org/abs/2308.16824