Google's New AI Agent SIMA Uses Human gameplay to Learn New Video Games
-
Google released a new AI agent, SIMA, that can follow instructions to complete tasks in video games, even ones it has never played before.
-
SIMA was trained using imitation learning to play games by watching human gameplay recordings. It can understand instructions like "turn left" and "open the map."
-
Researchers evaluated SIMA on 600 gaming skills and found it transfers abilities between games and outperforms game specialists.
-
SIMA still makes some mistakes in understanding precise instructions, like which specific tree to chop down.
-
DeepMind sees video games as a good training environment for AI systems to learn real world concepts like navigation and object interaction.