Phi-1.5: big-model reasoning in just 1.3 billion parameters
In one sentence Microsoft Research shows that 1.3B parameters trained on 'textbook quality' synthetic data produce multi-step reasoning comparable to models five times larger.
In the AI world there has always been a belief that being intelligent requires being large. Tens or hundreds of billions of parameters, trained on all available internet text.
Microsoft Research proved this isn't necessarily true. Phi-1.5 has only 1.3 billion parameters — as small as a 2020-era model — yet handles multi-step reasoning like 7 billion parameter models.
The secret? Not training it on the whole internet, but on texts written specifically with the quality of a good school textbook: clear, structured, full of step-by-step reasoning examples. It's the difference between studying on Wikipedia versus studying from an excellent school manual.
Companies
Microsoft
Tools
—
Tags
Sources