GPT-Neo: the first open source clone of GPT-3
In one sentence EleutherAI releases GPT-Neo 1.3B and 2.7B, open source language models trained on The Pile — the first serious attempt to replicate the GPT-3 architecture with public weights.
A grassroots collective of researchers called EleutherAI, born on Discord, releases GPT-Neo: a GPT-3-like language model with weights you can download for free.
Until that point large language models were locked inside companies: you paid the API or nothing. GPT-Neo is the first real crack in that wall. It isn't as big as GPT-3, but for anyone who wants to study, experiment, or build on top without depending on a company, it's the first real option.
The whole open source wave that leads to Llama, Mistral, Mixtral and DeepSeek starts here.
Companies
EleutherAI
Tools
GPT-Neo
Tags
Sources