In this study, researchers explore whether transformers can develop the ability to reason implicitly over parametric knowledge, a challenging skill for language models. They discover that transformers can indeed learn implicit reasoning through extensive training known as “grokking.” The research highlights varying levels of generalization across different reasoning types, with transformers struggling to generalize systematically for composition but succeeding for comparison. By analyzing the model’s internal mechanisms, the study provides insights into how grokking works and its connection to systematicity. Surprisingly, the study shows that a fully grokked transformer outperforms models based on non-parametric memory for complex reasoning tasks.
https://arxiv.org/abs/2405.15071