Reading path
AI Researcher & Academic
Foundational papers, architectures, scaling laws, reasoning, and Nobel prizes: the scientific thread.
You are a researcher, PhD student or academic who wants to reconstruct the scientific trajectory of modern AI through the contributions that actually moved the frontier. This path connects foundational papers on architectures and scaling with breakthroughs in reasoning and the institutional recognition that has marked the scientific legitimation of the field.
- 01
Why it matters to you
The canonical paper on in-context few-shot learning: it demonstrates for the first time that parameter scaling produces emergent capabilities not anticipated by training tasks.
Landmark Foundation ModelsGPT-3: the paper that opens the scaling-laws era
OpenAI publishes 'Language Models are Few-Shot Learners' and shows that at 175B parameters a model learns new tasks from a handful of examples in the prompt.
- 02
Why it matters to you
The Stanford report that codified the foundation model concept as a unifying paradigm: essential for framing the theoretical debate on models' generalist capabilities.
High Foundation ModelsOn the Opportunities and Risks of Foundation Models: Stanford coins the term
Stanford's Center for Research on Foundation Models publishes a 200+ page report coining the term foundation models, now standard in technical, academic and regulatory discourse.
- 03
Why it matters to you
Chinchilla laws redefine the optimal ratio between parameters and training tokens: one of the most-cited empirical results of the modern era, correcting the intuitions of Kaplan et al.
Landmark Foundation ModelsChinchilla: the big models were undertrained
DeepMind publishes the Chinchilla paper and shows that, given equal compute, smaller models trained on far more tokens beat oversized undertrained ones.
- 04
Why it matters to you
Constitutional AI introduces alignment via explicit principles and self-critique: a foundational methodological contribution to the debate on RLHF and model safety.
Medium AI SecurityConstitutional AI: the model self-corrects without humans in the loop
Anthropic publishes Constitutional AI: instead of pure RLHF, the model critiques and revises its own responses following a written 'constitution'. Less human labeling, more transparency.
- 05
Why it matters to you
o1 brings extended chain-of-thought at runtime as a reasoning strategy: the empirical result that reopened the academic debate on the separation of System 1 and System 2 in AI.
Landmark Foundation Modelso1: the first model that 'thinks before answering'
OpenAI ships o1-preview and o1-mini: models trained with RL on reasoning chains. On math, physics, competitive coding they beat GPT-4o by a huge margin. Paradigm shift.
- 06
Why it matters to you
S1 shows that test-time compute can be controlled with an explicit budget: a reproducible result that raises methodological questions about reasoning benchmarks.
High Foundation Modelss1: 1000 examples and a prompt trick to replicate a reasoning model
Stanford/UW paper: with 1000 curated examples and a technique called 'budget forcing' they fine-tune Qwen2.5-32B to compete with o1-preview on math. Training cost: <$50.
- 07
Why it matters to you
The Nobel Prize to Hinton for artificial neural networks marks the institutional legitimation of deep learning: a moment to reflect on the disciplinary history of the field.
Landmark Foundation Models2024 Nobel Prize in Physics to Hopfield and Hinton for artificial neural networks
The Royal Swedish Academy awards the 2024 Physics Nobel to John Hopfield and Geoffrey Hinton for their foundational work on artificial neural networks, formally recognizing AI as a discipline.