vLLM v0.7: the new V1 engine arrives in alpha
In one sentence On January 27, 2025 vLLM ships v0.7.0 and previews the rewritten V1 engine in alpha, enabled with VLLM_USE_V1=1: unified scheduler with chunked prefill, prefix caching on by default, torch.compile and up to 1.7x throughput over V0.
vLLM is one of the most widely used open-source engines for running language models on your own servers. With version 0.7.0, released on January 27, 2025, the project previews (in alpha) a rewritten internal engine called V1.
The new engine is not the default yet: you turn it on with an environment variable, without changing application code. The developers claim up to 1.7 times the throughput of the previous engine, mainly thanks to lower CPU overhead.
In V1, reading the prompt is split into chunks and mixed with generation (chunked prefill), and reuse of already-computed prefixes (prefix caching) is always on. Some features, such as LoRA, speculative decoding and pipeline parallelism, are not yet supported in this alpha.
Companies
vLLM Project
Tools
vLLM v0.7, vLLM V1 engine
Tags
Sources