Mixture of Denoisers
A pretraining strategy (UL2, Google 2022) that trains a single model on multiple denoising objectives simultaneously: left-to-right language modeling, span prediction (BERT-style masked spans of varying lengths and corruptions), and prefix language modeling. Unifies the strengths of GPT-style and T5-style pretraining. The model learns when to use each mode based on a sentinel token that signals the objective type.
In practice
A researcher wanting a flexible model for both completion and question answering can use UL2 or a Flan-UL2 checkpoint without choosing between encoder-decoder (T5) and decoder-only (GPT) architectures. In practice, the sentinel token `[S2S]`, `[NLU]`, or `[NLG]` must be prepended to the prompt to activate the correct mode — a detail that significantly impacts performance and is often omitted, causing poor results.
Related terms
Seen in the wild
105 entries mentioning it- MediumMCP TypeScript SDK: a shared server delivered one client's responses to another (CVE-2026-111770)
- HighMistral Large 4 enters public preview: 1T-parameter multimodal MoE, weights promised by end of October
- MediumCloudflare launches Clef and Clef-flash, decision models with typed outputs and probabilities
- MediumBlack Forest Labs publishes FLUX 3 Action weights, a 7B world action model for robots
- HighAnthropic releases Claude Opus 5.5: lower price and first place on the Artificial Analysis index
- HighXiaomi opens MiMo-V2.6 under MIT: the 1.02T-parameter Pro leads open weights
- HighDeepSeek V4.1 Flash: 552B open MoE under MIT with KV cache cut to a quarter
- MediumGitHub HydraFusion: Copilot CLI orchestrates several models per task
- HighZ.ai opens GLM-5.3-Flash: mystery model Ox Alpha revealed, MIT-licensed 320B MoE
- HighOpenAI launches the GPT-5.6 family in three variants: Luna, Terra, and Sol
- LandmarkAnthropic launches Claude Fable 5, its first Mythos-class model for everyone
- HighPhysical Intelligence π0.7: a generalist robot model that starts recombining skills
- HighMistral Small 4: reasoning, vision and agentic coding in a single open-weight model
- MediumNano Banana 2: Google rebuilds its image model around consistency and text
- HighAlibaba releases Qwen 3.5: sparse multimodal MoE, 262K context, Apache 2.0
- HighMCP turns one: 2025-11-25 spec, registry near 2,000 servers, MCP Apps proposal
- HighGemini 3 Pro: Google launches its new flagship, in Search from day one
- MediumClaude Haiku 4.5: the small model that comes close to Sonnet 4 at coding
- LandmarkGPT-5: OpenAI puts a fast model and a reasoning model behind an automatic router
- HighLM Studio 0.3.17 becomes an MCP host: external tools for local models