The author discusses the potential timeline for achieving artificial general intelligence (AGI) based on the scalability of language models. They suggest that if scaling continues to improve performance, we could see powerful AIs capable of automating cognitive labor by 2040 or even sooner. However, if scaling doesn’t work, the path to AGI becomes much more difficult. The author presents a debate between a believer and a skeptic on the effectiveness of scaling and the challenges it poses. The skeptic raises concerns about the lack of high-quality language data and the limitations of self-play and synthetic data approaches. Meanwhile, the believer emphasizes the ease of producing impressive AI models through scaling and the potential for synthetic data to work. The author also highlights the ongoing progress in language models, their ability to develop internal representations, and their potential for enhancing reasoning capabilities. Overall, the article presents contrasting viewpoints on the feasibility of scaling language models for AGI development.
https://www.dwarkeshpatel.com/p/will-scaling-work