The study challenges the belief that complex reasoning tasks require extensive training data, showing that mathematical reasoning abilities can be achieved with just 817 examples. The model, LIMO, outperforms previous models on mathematical reasoning tasks while using only 1% of the training data. It also demonstrates exceptional generalization across diverse benchmarks, suggesting that less training data may lead to better generalization. The Less-Is-More Reasoning Hypothesis proposes that sophisticated reasoning capabilities can emerge in models with comprehensive domain knowledge encoded during pre-training, with minimal post-training examples serving as cognitive templates. The open-source suite, LIMO, is available for further research.
https://arxiv.org/abs/2502.03387