Gemini Robotics 1.5: DeepMind makes robots plan and reason before acting
In one sentence Google DeepMind unveils Gemini Robotics 1.5 (vision-language-action model) and Gemini Robotics-ER 1.5 (embodied reasoning and planning). ER 1.5 is available to developers in the Gemini API via Google AI Studio; 1.5 only to select partners. Tasks learned on ALOHA 2 also work on a bi-arm Franka and the Apollo humanoid without retraining.
Google DeepMind unveils two models that work as a pair to let robots carry out multi-step tasks.
Gemini Robotics-ER 1.5 acts as the "planning brain": it understands the space around the robot, estimates task progress and can use tools such as Google Search or user-defined functions. It then passes natural-language instructions, one step at a time, to Gemini Robotics 1.5.
Gemini Robotics 1.5 is the model that turns images and instructions into motor commands. What is new is that it reasons before moving and can explain in natural language what it is doing.
The most interesting result is transfer across different robots: tasks taught only to the two-armed ALOHA 2 robot also worked on the bi-arm Franka and on Apptronik's Apollo humanoid without retraining, and vice versa. ER 1.5 is available to developers through the Gemini API in Google AI Studio, while Gemini Robotics 1.5 remains limited to select partners.
Companies
Google DeepMind
Tools
Gemini Robotics 1.5, Gemini Robotics-ER 1.5
Tags
Sources