IP-Adapter: an image as a prompt for Stable Diffusion
In one sentence Tencent AI Lab releases IP-Adapter, a 22M-parameter adapter that gives text-to-image diffusion models the ability to use an image as a prompt, without modifying the base model. Code and models on GitHub under Apache 2.0.
Getting the desired image from text alone often takes a lot of prompt engineering. Alternatives based on fine-tuning the model need large computing resources and don't transfer easily to other models.
IP-Adapter is a small add-on module that lets you use a reference image as a prompt, alone or together with text. The diffusion model stays frozen: only the adapter is trained.
That is why it also works on custom models derived from the same base model and combines with existing controllable generation tools. Code and models for Stable Diffusion 1.5 come out on August 16, 2023, and for SDXL 1.0 on August 18.
Companies
Tencent AI Lab
Tools
IP-Adapter, Stable Diffusion
Tags
Sources