DALL·E and CLIP: OpenAI connects text and images
In one sentence On January 5, 2021 OpenAI introduces DALL·E, a 12-billion-parameter transformer that generates images from text descriptions, together with CLIP, which learns to connect images and text from 400 million web-scraped pairs.
OpenAI announces two models on the same day: DALL·E, which draws images from a written description, and CLIP, which figures out which caption best describes an image.
DALL·E is not made available to the public, but the examples shown (an armchair shaped like an avocado, a baby daikon radish in a tutu walking a dog) spread across the web. CLIP, by contrast, is published with code and weights, and soon becomes a component of much later research.
The two work together: CLIP is also used to pick, among the images DALL·E generates, the ones that best match the text.
Companies
OpenAI
Tools
DALL-E, CLIP
Tags
Sources