Whisper large v3 turbo
Model Overview¶
Whisper is a pre-trained model for automatic speech recognition (ASR) and speech translation. Trained on >5M hours of labeled data, Whisper demonstrates a strong ability to generalise to many datasets and domains in a zero-shot setting.
- Model Architecture:Whisper large-v3-turbo is a finetuned version of a pruned Whisper large-v3. In other words, it's the exact same model, except that the number of decoding layers have reduced from 32 to 4
- Model Source: openai/whisper-large-v3-turbo
- License: Apache 2.0 license
QPC Configurations¶
| Precision | SoCs / Tensor slicing | NSP-Cores (per SoC) | Full Batch Size | Chunking Prompt Length | Context Length (CL) | CCL_Enabled | QPC URL | QPC Size | QPC Download | Onnx URL | Onnx Download | Generation Date |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| MXFP6 | 1 | 16 | 1 | 1 | 448 | False | https://dc00tk1pxen80.cloudfront.net/SDK1.21.6/openai/whisper-large-v3-turbo/openai_whisper-large_v3_turbo_qpc_16cores_1bs_448cl_1device.tar.gz | 752M | Download | https://dc00tk1pxen80.cloudfront.net/SDK1.21.6/openai/whisper-large-v3-turbo/openai_whisper-large_v3_turbo_ONNx.tar.gz | Download | 12-Jun-2026 |
Run This Model¶
Download QPCs¶
mkdir -p openai/whisper-large-v3-turbo
cd openai/whisper-large-v3-turbo
# Download QPC
wget <QPC_Download_URL>
tar xzvf <qpc_filename.tar.gz>
Run QPC¶
export VLLM_QAIC_QPC_PATH=/path/to/qpc
python3 -m vllm.entrypoints.openai.api_server \
--model openai/whisper-large-v3-turbo \
--port <PORT> \
--device-group <DEVICE_IDS> \
--max-model-len <CTX_LEN> \
--max-seq-len-to-capture <PREFILL_SEQ_LEN> \
--max-num-seqs 1 \
--quantization mxfp6 \
--kv-cache-dtype mxint8 \
--override-qaic-config "num_cores:[num_cores] qpc_path:[qpc_path]"