DeepMind introduces a groundbreaking AI agent called SIMA that can understand natural-language instructions and carry out tasks in multiple video game settings. The research emphasizes the importance of using video games as learning environments for AI systems. SIMA was trained on nine different video games, displaying the ability to perform tasks from simple navigation to crafting. Unlike other AI agents, SIMA does not require access to a game’s source code and can interact with any virtual environment using only image and language inputs. The ultimate goal is to develop more general, language-driven AI agents that can effectively complete complex tasks in a variety of environments.
https://deepmind.google/discover/blog/sima-generalist-ai-agent-for-3d-virtual-environments/