High Multimodal AI · 1 min read
BLIP-2: the Q-Former bridge between vision and language
In one sentence Salesforce Research introduces BLIP-2: a lightweight Q-Former bridges a frozen visual encoder and a frozen LLM; on zero-shot VQAv2 it beats Flamingo80B by 8.7 points with 54x fewer trainable parameters.
Reading level
BLIP-2 is a Salesforce system that connects an image encoder and a large language model using an intermediate component called Q-Former. Both the visual model and the text model stay "frozen" — only the Q-Former gets trained. This dramatically reduces training costs. The result outperforms prior models on several vision-language tasks while updating far fewer parameters.
Companies
Salesforce
Tools
BLIP-2, Q-Former
Tags
BLIP-2Q-FormerImage CaptioningSalesforceEfficient Training
Sources