Microsoft RoboGen: generating robot tasks, skills and environments from text
In one sentence Microsoft and CMU introduce RoboGen: an automatic pipeline using LLMs to generate robotic tasks, simulated environments, and training skills from a simple text description.
Training a robot normally requires enormous amounts of data: you have to show the robot how to do each individual thing, thousands of times. RoboGen solves this problem differently: instead of collecting real data, it automatically generates training situations using a language model.
You describe in words what you want the robot to do (for example "open a drawer and pick up a bottle"), and RoboGen automatically creates the virtual environment, the sequence of steps, and the training code. The robot trains in the simulator and is then transferred to the real world.
This drastically reduces the time and cost of teaching new behaviors to a robot.
Companies
Microsoft, Carnegie Mellon University
Tools
RoboGen, GPT-4, Isaac Gym
Tags
Sources