Stable Video Diffusion 1.1: video from a single image with motion control

Generating a video starting from a single still image was the next frontier after image generation. Stable Video Diffusion 1.1 allows exactly this: provide a photo, and the model generates a short video clip coherent with the visual content.

The novelty over the previous version is motion control: you can indicate how "lively" the scene should animate, from nearly static to broader movements. This avoids the shaky and incoherent videos that plagued early attempts.

The model weights are freely available, which allowed the open-source community to quickly integrate it into tools like ComfyUI.