Google DeepMind's new AI lets robots learn by talking to themselves

Google DeepMind is developing an innovative AI system that endows robots with an "inner voice" or internal narration, allowing them to describe visual observations in natural language as they perform tasks. This approach, detailed in a recent patent filing, enables robots to link what they see with corresponding actions, facilitating "zero-shot" learning—where robots can understand and interact with unfamiliar objects without prior training. This method not only improves task learning efficiency but also reduces memory and computational requirements, enhancing robots' adaptability in dynamic environments. Building on this concept, DeepMind introduced Gemini Robotics On-Device, a compact vision-language model designed to run entirely on robots without cloud connectivity. This on-device model supports fast, reliable performance in latency-sensitive or offline contexts, such as healthcare, while maintaining privacy. Despite its smaller size, Gemini Robotics On-Device can perform complex tasks like folding clothes or unzipping bags with low latency and can adapt to new tasks with minimal demonstrations. Although it lacks built-in semantic safety features found in