Qwen-1.5: 0.5B-110B family with 32k context and 30+ languages
In one sentence Alibaba Cloud releases Qwen-1.5, a 0.5B-to-110B parameter family with native 32k context support, GQA, bilingual EN/ZH, instructions in 30+ languages, and RLHF chat.
Qwen-1.5 is an Alibaba model family that covers every need: from the tiny 500-million-parameter model for mobile devices, to the 110-billion model for enterprise use.
The main novelty is native support for 32,000 tokens of context, which allows processing long documents without extra tricks. It also supports over 30 languages, with Italian, Arabic, Korean, and many others well represented.
It was the first time a lab opened such a complete family with instructions in so many languages, marking a turning point for those working in languages other than English and Chinese.
Companies
Alibaba Cloud
Tools
Qwen-1.5
Tags
Sources