The OK-Robot framework focuses on language-conditioned pick-and-drop tasks in various homes, showcasing its abilities in 10 New York City homes through 171 tasks. Despite a 58.5% success rate in completely novel environments, the analysis reveals a range of failure causes, with retrieving the right object, difficult poses, and hardware issues being the top reasons. The detailed breakdown in the paper explains the failure modes and discusses common scenarios. The authors emphasize the importance of integrating open-knowledge models in robotics, providing an open-source code on GitHub for further exploration and development.
https://ok-robot.github.io/