Alibaba drops Qwen2-VL to take on GPT-4o in vision
Alibaba just released Qwen2-VL, open-sourcing the 2B and 7B models under Apache 2.0 while offering its flagship 72B version via API. The model handles dynamic resolutions, parses 20-minute videos, and powers mobile device agents. Why it matters: The 72B flagship rivals GPT-4o on visual benchmarks, bringing top-tier multimodal power to open weights and developers.
Qwen2-VL is an open-source vision-language model designed to analyze images of any size, digest videos over 20 minutes long, and act as a visual agent capable of operating devices like phones.
Alibaba has released the weights for both the 2B and 7B models under an Apache 2.0 license, while offering its flagship 72B model via API on DashScope. The code is already integrated into Hugging Face Transformers and vLLM.
What it can do
Compared to its predecessor, Qwen2-VL drops fixed-size image constraints. It handles odd aspect ratios, distorted charts, and messy handwritten notes without choking on the layout.
Video understanding gets a serious bump. The model can process videos exceeding 20 minutes, allowing users to ask questions about specific clips, extract summaries, or run live conversations over footage.
It also functions as a visual agent. Qwen2-VL can interpret mobile UI screens, make decisions based on what is displayed, and trigger function calls to check real-time data like flights or packages based on visual cues.
Benchmark-wise, Alibaba reports that the 72B model rivals GPT-4o and Claude 3.5-Sonnet across visual reasoning benchmarks like MathVista and DocVQA. The 7B model offers a solid lightweight alternative, while the compact 2B model is optimized for mobile and edge setups.
How it works
Most traditional vision models force images into static square grids, which warps aspect ratios and blurs high-res text. Qwen2-VL uses Naive Dynamic Resolution, converting images into a variable number of tokens to match their native dimensions.
Under the hood, it pairs a 600M-parameter Vision Transformer with Qwen2 language backbones. It introduces Multimodal Rotary Position Embedding (M-ROPE), splitting position data across time, height, and width to process 1D text, 2D images, and 3D video inputs simultaneously.
There are real limits to keep in mind. Qwen2-VL cannot process audio tracks in video, its internal knowledge cutoff is June 2023, and it still struggles with exact counting, character recognition, and 3D spatial awareness.
How to run it
You can grab the 2B and 7B weights right now from Hugging Face and ModelScope. Because integration is fresh, you will need to install transformers from source and grab the qwen-vl-utils helper package for handling base64 and mixed inputs.
If you want the top-performing 72B model, it is available through Alibaba Cloud's DashScope via an OpenAI-compatible API.
Why it matters
We see Qwen2-VL as a major win for developers building visual agents and document processing pipelines. By combining dynamic resolution with Apache 2.0 licensing on the 7B and 2B models, it provides a high-performing local option that does not require sending private documents to proprietary APIs.
It is not a full multimodal model yet since it lacks native audio processing, but for dense document extraction, multilingual OCR, and video QA, it sets a formidable open-weight standard.
Sources (1)
- qwenlm.github.io(opens in a new tab)Originalqwenlm.github.io