Ai2's Tülu 3: a fully open post-training recipe
In one sentence The Allen Institute for AI (Ai2) releases Tülu 3, 8B and 70B models built on Llama 3.1 base with SFT, DPO and the new RLVR; it publishes data, code, recipes and evaluations. According to Ai2 it surpasses the Instruct versions of Llama 3.1, Qwen 2.5 and Mistral.
Meta releases Llama in two versions: "base" (weights after pre-training) and "Instruct" (refined to follow instructions). The step between them, post-training, is the least documented part of model development.
On November 21, 2024 the Allen Institute for AI (Ai2) publishes Tülu 3: it starts from the Llama 3.1 8B and 70B bases and applies a fully public post-training recipe. It releases the models from each stage, the datasets, the training code, the recipes and the evaluation system.
According to the paper, Tülu 3 surpasses the Instruct versions of Llama 3.1, Qwen 2.5 and Mistral, and even closed models such as GPT-4o-mini and Claude 3.5 Haiku. Anyone can reproduce or adapt the procedure.
Companies
Allen Institute for AI
Tools
Tulu 3
Tags
Sources