Latent Consistency Models: Stable Diffusion images in 2-4 steps
In one sentence Tsinghua University researchers present Latent Consistency Models: distilling a latent diffusion model such as Stable Diffusion yields 768x768 images in 2-4 inference steps instead of dozens, with about 32 A100 GPU hours of training.
A diffusion model like Stable Diffusion generates an image by removing noise over many successive steps, and every step is a full pass through the network. That is why generation is slow.
Latent Consistency Models (LCM) start from an already trained model and distill it: a student model learns to predict the final result of the process directly, skipping the intermediate steps. That way 2-4 steps are enough to produce an image.
According to the authors, training is cheap (about 32 A100 GPU hours for the 768x768 model) and the few-step results are state of the art. The method also includes a technique for adapting LCMs to custom image datasets.
Companies
Tsinghua University
Tools
—
Tags
Sources