Eureka: NVIDIA uses GPT-4 to write reward functions and train expert robots
In one sentence NVIDIA presents Eureka, the first system to use an LLM (GPT-4) to automatically generate reward functions for robotic reinforcement learning. The system achieves expert-level dexterous manipulation, including pen spinning, without manual reward design.
In reinforcement learning, teaching a robot what to do requires writing a "reward function": a mathematical formula that tells the robot how well it is doing. Designing this formula is artisanal work requiring weeks of experience and experimentation. NVIDIA's Eureka eliminates this effort.
The system uses GPT-4 to read the simulator code, understand what the robot needs to do, and automatically write the reward function in Python. That function is then tested in the Isaac Gym simulator, the result is shown back to GPT-4, and the model rewrites and improves the function. This cycle repeats until excellent results are achieved.
The most impressive test: spinning a pen between fingers like a juggler. This is an extremely difficult dexterous manipulation task. Eureka achieves it at expert human level, using rewards generated by an LLM with no manual intervention in the design.
The breakthrough is that designing complex robotic behaviors no longer requires an RL expert spending weeks calibrating mathematical formulas. Just describe the goal.
Companies
NVIDIA
Tools
—
Tags
Sources