Qwen2.5 VL 32B Instruct
Model Overview¶
Qwen2.5-VL is a vision-language model with the below Key Enhancements than earlier models: Understand things visually: Qwen2.5-VL is not only proficient in recognizing common objects such as flowers, birds, fish, and insects, but it is highly capable of analyzing texts, charts, icons, graphics, and layouts within images.
Being agentic: Qwen2.5-VL directly plays as a visual agent that can reason and dynamically direct tools, which is capable of computer use and phone use.
Understanding long videos and capturing events: Qwen2.5-VL can comprehend videos of over 1 hour, and this time it has a new ability of cpaturing event by pinpointing the relevant video segments.
Capable of visual localization in different formats: Qwen2.5-VL can accurately localize objects in an image by generating bounding boxes or points, and it can provide stable JSON outputs for coordinates and attributes.
Generating structured outputs: for data like scans of invoices, forms, tables, etc. Qwen2.5-VL supports structured outputs of their contents, benefiting usages in finance, commerce, etc.
- Model Architecture: Dynamic Resolution and Frame Rate Training for Video Understanding: Extended dynamic resolution to the temporal dimension by adopting dynamic FPS sampling, enabling the model to comprehend videos at various sampling rates. Accordingly, we update mRoPE in the time dimension with IDs and absolute time alignment, enabling the model to learn temporal sequence and speed, and ultimately acquire the ability to pinpoint specific moments. Streamlined and Efficient Vision Encoder: We enhance both training and inference speeds by strategically implementing window attention into the ViT. The ViT architecture is further optimized with SwiGLU and RMSNorm, aligning it with the structure of the Qwen2.5 LLM.
- Model Source: Qwen/Qwen2.5-VL-32B-Instruct
- License: apache-2.0
Multi Model QPC Configuration - Using QEfficient # 1¶
| Precision | SoCs / Tensor slicing | NSP-Cores (per SoC) | Full Batch Size | Chunking Prompt Length | Context Length (CL) | CCL_Enabled | QPC URL | QPC Size | QPC Download | Onnx URL | Onnx Download | Generation Date |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| MXFP6 | 2 | 8 | 1 | 128 | 8192 | False | https://dc00tk1pxen80.cloudfront.net/SDK1.21.6/Qwen/Qwen2.5-VL-32B-Instruct/Qwen2.5_VL_32B_Instruct_Encoder_qpc_8cores_128pl_8192cl_2devices_mxfp6_mxint8.tar.gz | 1.4GB | Download | https://dc00tk1pxen80.cloudfront.net/SDK1.21.6/Qwen/Qwen2.5-VL-32B-Instruct/Qwen2.5-VL-32B-Instruct_Encoder_ONNX.tar.gz | Download | 17-June-2026 |
| MXFP6 | 2 | 8 | 1 | 128 | 8192 | False | https://dc00tk1pxen80.cloudfront.net/SDK1.21.6/Qwen/Qwen2.5-VL-32B-Instruct/Qwen2.5_VL_32B_Instruct_Decoder_qpc_8cores_128pl_8192cl_2devices_mxfp6_mxint8.tar.gz | 26GB | Download | https://qualcom-qpc-models.s3-accelerate.amazonaws.com/SDK1.21.6/Qwen/Qwen2.5-VL-32B-Instruct/Qwen2.5-VL-32B-Instruct_Decoder_CCL_Disabled_ONNX.tar.gz | Download | 17-June-2026 |
Multi Model QPC Configuration - Using QEfficient # 2¶
| Precision | SoCs / Tensor slicing | NSP-Cores (per SoC) | Full Batch Size | Chunking Prompt Length | Context Length (CL) | CCL_Enabled | QPC URL | QPC Size | QPC Download | Onnx URL | Onnx Download | Generation Date |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| MXFP6 | 4 | 8 | 1 | 128 | 65536 | True | https://dc00tk1pxen80.cloudfront.net/SDK1.21.6/Qwen/Qwen2.5-VL-32B-Instruct/Qwen2.5_VL_32B_Instruct_Encoder_qpc_8cores_128pl_[2048,4096,8192,12288,16384,24576,32768]ccl_4devices_mxfp6_mxint8.tar.gz | 1.4GB | Download | https://dc00tk1pxen80.cloudfront.net/SDK1.21.6/Qwen/Qwen2.5-VL-32B-Instruct/Qwen2.5-VL-32B-Instruct_Encoder_ONNX.tar.gz | Download | 17-June-2026 |
| MXFP6 | 4 | 8 | 1 | 128 | 65536 | True | https://dc00tk1pxen80.cloudfront.net/SDK1.21.6/Qwen/Qwen2.5-VL-32B-Instruct/Qwen2.5_VL_32B_Instruct_Decoder_qpc_8cores_128pl_[2048,4096,8192,12288,16384,24576,32768,65536]ccl_4devices_mxfp6_mxint8.tar.gz | 31GB | Download | https://qualcom-qpc-models.s3-accelerate.amazonaws.com/SDK1.21.6/Qwen/Qwen2.5-VL-32B-Instruct/Qwen2.5-VL-32B-Instruct_Decoder_CCL_Enabled_ONNX.tar.gz | Download | 17-June-2026 |
Multi Model QPC Configuration - Using QEfficient # 3¶
| Precision | SoCs / Tensor slicing | NSP-Cores (per SoC) | Full Batch Size | Chunking Prompt Length | Context Length (CL) | CCL_Enabled | QPC URL | QPC Size | QPC Download | Onnx URL | Onnx Download | Generation Date |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| MXFP6 | 8 | 16 | 1 | 128 | 8192 | False | https://dc00tk1pxen80.cloudfront.net/SDK1.21.6/Qwen/Qwen2.5-VL-32B-Instruct/Qwen--Qwen2.5-VL-32B-Instruct_Encoder_qpc_16cores_1fbs_128pl_8192cl_8devices_sdk_1_21_6.tar.gz | 1.5GB | Download | https://dc00tk1pxen80.cloudfront.net/SDK1.21.6/Qwen/Qwen2.5-VL-32B-Instruct/Qwen2.5-VL-32B-Instruct_Encoder_ONNX.tar.gz | Download | 30-June-2026 |
| MXFP6 | 8 | 16 | 1 | 128 | 8192 | False | https://dc00tk1pxen80.cloudfront.net/SDK1.21.6/Qwen/Qwen2.5-VL-32B-Instruct/Qwen--Qwen2.5-VL-32B-Instruct_Decoder_qpc_16cores_1fbs_128pl_8192cl_8devices_sdk_1_21_6.tar.gz | 48GB | Download | https://qualcom-qpc-models.s3-accelerate.amazonaws.com/SDK1.21.6/Qwen/Qwen2.5-VL-32B-Instruct/Qwen2.5-VL-32B-Instruct_Decoder_CCL_Disabled_ONNX.tar.gz | Download | 30-June-2026 |
Multi Model QPC Configuration - Using disagg_serving¶
| Precision | SoCs / Tensor slicing | NSP-Cores (per SoC) | Full Batch Size | Chunking Prompt Length | Context Length (CL) | CCL_Enabled | QPC URL | QPC Size | QPC Download | Onnx URL | Onnx Download | Generation Date |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| MXFP6 | 1 | 8 | 1 | 512 | 4096 | False | https://dc00tk1pxen80.cloudfront.net/SDK1.21.6/Qwen/Qwen2.5-VL-32B-Instruct/Qwen_Qwen_2_5_vl_32B_Encoder_qpc_8cores_1bs_512pl_4096cl_1devices_sdk1_21_6.tar.gz | 1.4GB | Download | https://dc00tk1pxen80.cloudfront.net/SDK1.21.6/Qwen/Qwen2.5-VL-32B-Instruct/Qwen_Qwen_2_5_vl_32B_Encoder_ONNX_sdk1_21_6.tar.gz | Download | 16-July-2026 |
| MXFP6 | 8 | 8 | 1 | 512 | 4096 | False | https://dc00tk1pxen80.cloudfront.net/SDK1.21.6/Qwen/Qwen2.5-VL-32B-Instruct/Qwen_Qwen_2_5_vl_32B_Decoder_qpc_8cores_1bs_512pl_4096cl_8devices_sdk1_21_6.tar.gz | 48GB | Download | https://qualcom-qpc-models.s3-accelerate.amazonaws.com/SDK1.21.6/Qwen/Qwen2.5-VL-32B-Instruct/Qwen_Qwen_2_5_vl_32B_Decoder_ONNX_sdk1_21_6.tar.gz | Download | 16-July-2026 |
Run This Model¶
Download QPCs¶
mkdir -p Qwen/Qwen2.5-VL-32B-Instruct
cd Qwen/Qwen2.5-VL-32B-Instruct
# Download Encoder QPC
wget <Encoder_QPC_Download_URL>
tar xzvf <encoder_qpc_filename.tar.gz>
# Download Decoder QPC
wget <Decoder_QPC_Download_URL>
tar xzvf <decoder_qpc_filename.tar.gz>
# Download Inference Script
wget http://qualcom-qpc-models.s3-website-us-east-1.amazonaws.com/QPC/multimodel_inference_1_21_6.py
Run QPC for models compiled using QEfficient¶
Replace <encoder_qpc_path> and <decoder_qpc_path> with the actual extracted QPC directories.
python3 multimodel_inference_1_21_6.py \
--model-id Qwen/Qwen2.5-VL-32B-Instruct \
--vision-qpc <encoder_qpc_path> \
--lang-qpc <decoder_qpc_path> \
--ctx-len <ctx_len> \
--prefill-seq-len 128 \
--device-ids <device_ids> \
--generation-len 200 \
--image-url "<image_url>" \
--prompt "<prompt>"
Run QPC for models compiled using disagg_serving¶
Replace <encoder_qpc_path> and <decoder_qpc_path> with the actual extracted QPC directories.
python3 -m qaic_disagg \
--encode-port 4912 \
--encode-device-group 0:1 \
--decode-port 4916 \
--decode-device-group 8:16 \
--port 4921 \
--model "Qwen/Qwen2.5-VL-32B-Instruct" \
--prefill-max-seq-len-to-capture 512 \
--decode-max-seq-len-to-capture 512 \
--max-model-len 4096 \
--encode-override-qaic-config "height=364 width=532 num_cores=8 num_frames=5 kv_offload=True qpc_path=<encoder_qpc_path>" \
--decode-override-qaic-config "height=364 width=532 num_cores=8 num_frames=5 kv_offload=True qpc_path=<decoder_qpc_path>" \
--encode-max-num-seqs 1 \
--decode-max-num-seqs 1 \
--quantization mxfp6 \
--kv-cache-dtype mxint8 \
--proxy_worker 3 \
--limit-mm-per-prompt '{"image":5}' \
--enable-prefix-caching \
--show-hidden-metrics-for-version=0.7 \
--router-policy least_outstanding \
--build-grafana-json