In practice
It is now a pillar of modern training: big models produce examples to train smaller ones (distillation) or to cover rare cases. It must be filtered carefully, because generator errors compound in the final model. Nvidia, Meta, and Anthropic use it heavily.
Related terms
Seen in the wild
5 entries mentioning it- HighNVIDIA Isaac GR00T N1: the open foundation model for humanoid robots unveiled at GTC
- MediumNemotron-4 340B: NVIDIA's model family for generating synthetic training data
- MediumRoboGen: an agent that generates its own tasks, scenes and supervision to train robots in simulation
- HighPhi-1.5: 1.3 billion parameters trained on textbook-style synthetic data
- HighPhi-1: 1.3 billion parameters and 50.6% on HumanEval