InternLM is introducing its open-sourced 7 billion parameter base model and chat model. The models have impressive features, such as leveraging trillions of high-quality tokens for training and supporting an 8k context window length for stronger reasoning capabilities. The toolset provided allows users to build their own workflows, and a lightweight training framework supports model pre-training without extensive dependencies. InternLM achieves remarkable performance optimizations, with nearly 90% acceleration efficiency during training on 1024 GPUs. The evaluation results showcase the model’s capabilities in various dimensions. The models are available in two formats for further use. However, it’s important to note that the model may produce unexpected outputs due to its size and probabilistic generation paradigm, including biases or harmful content. The code and weights are open-source, and users are encouraged to contribute and provide feedback. Commercial use requires written permission from the official team.
https://github.com/InternLM/InternLM