The author discusses how reasoning is implicit in written text and conversation, using the Self-Taught Reasoner (STaR) model as an example. They introduce Quiet-STaR, where language models learn to generate rationales to explain future text, improving predictions. The article addresses challenges such as computational cost and the need to predict beyond individual tokens. Through continued pretraining, the Quiet-STaR model shows zero-shot improvements on various tasks, highlighting the potential for language models to learn to reason in a more general and scalable way. The approach marks a significant step forward in enhancing the LM’s ability to answer difficult questions without fine-tuning.
https://arxiv.org/abs/2403.09629