Gemini 1.5 Pro: 1 million tokens in context
In one sentence Google announces Gemini 1.5 Pro: Mixture of Experts architecture, 128K standard context, 1M in preview. New benchmark: near-perfect 'needle in a haystack' retrieval over long inputs.
Google ships Gemini 1.5 Pro, and the announcement is dominated by one number: 1 million tokens of context. In preview you can load an hour of video, 11 hours of audio, a 30,000-line codebase into a single prompt.
Having a wide context isn't enough — you also have to actually use it: Google shows "needle in a haystack" tests — hide a specific sentence in a million tokens, then ask the model to find it. Result: 99% accuracy.
From this moment, "context window" stops being a limit and becomes a design tool.
Companies
Tools
Gemini 1.5 Pro
Tags
Sources