OpenAI o3: the preview that leaps on ARC-AGI
In one sentence On the last day of the '12 Days of OpenAI' OpenAI previews o3 and o3-mini: 87.5% on ARC-AGI with high compute, 25.2% on FrontierMath, 71.7% on SWE-bench Verified. No public release, only safety testing.
On December 20, 2024, the last day of the "12 Days of OpenAI", Sam Altman presents o3, successor to o1, together with the smaller o3-mini. The name "o2" is skipped to avoid a conflict with the British telecom operator O2. It is not a release: the models only enter safety testing, with access on request for researchers.
The most discussed result is on ARC-AGI, a reasoning test on novel visual puzzles that earlier models barely solved (GPT-4o stopped at 5%). In tests run by the ARC Prize team, o3 scores 75.7% with a modest compute budget and 87.5% with about 172 times more compute, at a cost of thousands of dollars per task.
François Chollet, creator of the benchmark, calls it a genuine breakthrough, while believing o3 is not AGI yet: it still fails some very easy tasks.
Companies
OpenAI
Tools
o3, o3-mini
Tags
Sources