Whisper large-v3-turbo locally: 809M parameters, 8x faster, with whisper.cpp and faster-whisper
In one sentence OpenAI released large-v3-turbo: 809M parameters against large-v3's 1.55B, obtained by cutting the decoder from 32 layers to 4 and fine-tuning for two epochs on multilingual transcription data only. Roughly 8x faster and about 6 GB of VRAM, with quality close to large-v2 but no translation support and clear degradation on Thai and Cantonese.
On 30 September 2024 OpenAI published a lighter version of its speech transcription model: Whisper large-v3-turbo. The problem it solves is concrete — large-v3 was accurate but slow, and transcribing an hour of meeting audio on an ordinary PC took an awkward amount of time.
The trick is entirely structural. Whisper works in two stages: first it listens to the audio (the encoder), then it writes out the text (the decoder). OpenAI left the listening half intact and cut the writing half from 32 layers down to 4 — the same as the smallest model in the family — then fine-tuned the result for two more training passes on transcription data alone. The model goes from 1.55 billion to 809 million parameters and runs roughly eight times faster, needing about 6 GB of video memory.
The trade-off is real and documented. Quality is comparable to the previous generation, large-v2, on most languages, but it gets noticeably worse on Thai and Cantonese. And critically: turbo was not trained on translation data, so it is not the tool for translating audio into another language — only for transcribing it in the language spoken.
For anyone who wants to keep audio in-house (meetings, calls, dictation) this is the variant that makes local transcription practical: it runs on a consumer GPU and needs no external service.
Companies
OpenAI
Tools
Whisper large-v3-turbo, whisper.cpp, faster-whisper
Tags
Sources