Robots worldwide are collaborating to share data on object manipulation with the goal of creating a general-purpose robotic brain. While generative AI tools like ChatGPT and Midjourney have revolutionized language models, enabling them to fulfill various user requests, the same success has not translated into robotics. Unlike the abundant text and image data available on the internet, robotic-interaction data is limited and created slowly by researchers for specific tasks. To overcome this, the RT-X project was launched, pooling together the experiences of many robots to train new robots. The initial phase of the project showed promising results, with a single unified model outperforming individual laboratory systems. The project also explored incorporating internet-sourced data for complex semantic reasoning tasks. The combination of internet data and multirobot data demonstrated improved performance in logical inferences and spatial reasoning. The RT-X project aims to continue its collaboration efforts, expand the dataset, and develop reusable models and techniques to transform the field of robotics.
https://spectrum.ieee.org/global-robotic-brain