Llama 3: 8B and 70B open competitive with Claude 3 Sonnet
In one sentence Meta releases Llama 3 in two initial sizes (8B, 70B). Trained on 15T tokens, improved tokenizer, 8K context. The 70B Instruct competes with Claude 3 Sonnet and Gemini 1.5 Pro on many benchmarks.
Meta ships Llama 3 in two sizes: 8 and 70 billion parameters. Same philosophy as Llama 2 (open + commercial license), but much stronger: the 70B model competes with top closed-source models.
The new piece: the training dataset is 15 trillion tokens (seven times Llama 2) and includes much more code and many more languages. Result: better at almost everything.
A 405B model is announced as "coming" (ships in July as Llama 3.1). For the open-source world, this is the moment the gap with closed-source truly closes.
Companies
Meta
Tools
Llama 3
Tags
Sources