Grade school teachers may not have taught you how to add 20-digit numbers, but it’s easier than you think. Instead of staring at the problem and writing down the answer, you need to walk through the problem step by step in the correct way. While chatbots like ChatGPT can ace simple arithmetic questions, they struggle with more complex problems. However, through chain-of-thought prompting, researchers have enabled language models like transformers to solve problems that were once out of reach. By studying the limitations of transformers using computational complexity theory, researchers are gaining insight into the intrinsic capabilities of these models and exploring new ways to build them effectively.
https://www.quantamagazine.org/how-chain-of-thought-reasoning-helps-neural-networks-compute-20240321/