Allen AI's Tülu 3: the first fully open post-training pipeline
In one sentence Allen Institute (AI2) releases Tülu 3: 8B/70B family with the first truly open post-training pipeline (code, data, recipes, eval), beating Llama 3.1 Instruct using only Meta's base.
Meta releases Llama in two versions: "base" (raw weights after initial training) and "Instruct" (refined to follow instructions). The pipeline to go from base to Instruct is one of the industry's best-kept secrets — Meta doesn't explain exactly how.
Allen AI, a non-profit research institute founded by Paul Allen, publishes Tülu 3: takes the Llama 3.1 base and applies a fully open post-training pipeline. Code, data, recipes, evals — all on GitHub.
Result: their Tülu 3 70B actually beats Llama 3.1 70B Instruct on many benchmarks. And anyone in a university or company can reproduce or adapt it.
Companies
Allen Institute for AI
Tools
Tulu 3
Tags
Sources