Code as Policies: language models that write robot control code
In one sentence Robotics at Google shows that LLMs trained on code completion can write Python robot-control programs from natural-language instructions, with no task-specific training, thanks to hierarchical code generation.
Instead of training a separate model for every robot task, Code as Policies uses a code-writing language model to generate Python programs that the robot executes. A natural-language instruction becomes code that calls existing perception functions and control APIs.
The key point is hierarchical generation: if the generated code uses a function that does not exist yet, the model defines it in turn, building complex solutions from simple blocks. A few examples in the prompt are enough, with no robot-specific training.
The approach was tested on several real robots: tabletop manipulation, drawing on a whiteboard, and a mobile robot that navigates and moves objects. It generalizes to new instructions and interprets vague terms like "faster" based on context.
Companies
Tools
Code as Policies
Tags
Sources