Qwen3 30B A3B Instruct 2507

Model Overview¶

Qwen3-30B-A3B-Instruct-2507 is a state-of-the-art large language model from the Qwen3 series, designed to provide strong performance across a wide range of domains including reasoning, coding, multilingual understanding, and long-context comprehension.

Qwen3-30B-A3B-Instruct-2507 brings the following improvements:

General Capabilities – Stronger instruction following, logical reasoning, mathematics, science, and coding, with better tool usage integration.
Knowledge Coverage – Substantial gains in handling long-tail knowledge and multiple languages.
Alignment – Enhanced alignment with user intent for open-ended and subjective tasks, producing more helpful and high-quality responses.
Extended Context – Native support for 256K tokens, making it highly capable for long-context tasks such as document understanding and multi-turn reasoning.
Efficient Mixture-of-Experts (MoE) – Designed with 128 experts, of which only 8 are active at a time, improving efficiency while maintaining large-model accuracy.

Model Architecture¶

Type: Causal Language Model (CLM)
Number of Parameters (Total): 30.5B
Number of Parameters (Non-Embedding): 29.9B
Activated Parameters: ~3.3B per forward pass (Mixture of Experts)
Number of Layers: 48
Number of Attention Heads (GQA): 32 for Q and 4 for KV
Number of Experts: 128 (8 activated per token)
Context Length: 262,144 tokens (256K) natively
Model Source: Qwen/Qwen3-30B-A3B-Instruct-2507
License: apache-2.0

QPC Configurations¶

Precision	SoCs / Tensor slicing	NSP-Cores (per SoC)	Full Batch Size	Chunking Prompt Length	Context Length (CL)	QPC URL	QPC Size	QPC Download	Onnx URL	Onnx Download	Generation Date
MXFP6	4	16	1	128	65536	https://dc00tk1pxen80.cloudfront.net/SDK1.21.4.0/Qwen/Qwen3-30B-A3B-Instruct-2507/Qwen_Qwen3-30B-A3B-Instruct-2507_qpc_16cores_128pl_65536cl_[1k,2k,4k,8k,16k,32k,64k]_1fbs_4devices_mxfp6_mxint8.tar.gz	55GB	Download	http://qualcom-qpc-models.s3-website-us-east-1.amazonaws.com/SDK1.21.4.0/Qwen/Qwen3-30B-A3B-Instruct-2507/Qwen_Qwen3-30B-A3B-Instruct-2507_ONNX.tar.gz	Download	30-Apr-2026

Run This Model¶

Download QPCs¶

mkdir -p Qwen/Qwen3-30B-A3B-Instruct-2507
cd Qwen/Qwen3-30B-A3B-Instruct-2507

# Download QPC
wget <QPC_Download_URL>
tar xzvf <qpc_filename.tar.gz>

Run QPC¶

Replace QPC_PATH with actual extracted QPC directories.

python3 -m vllm.entrypoints.openai.api_server \
  --host 0.0.0.0 \
  --port <PORT> \
  --model Qwen/Qwen3-30B-A3B-Instruct-2507 \
  --device-group <DEVICE_IDS> \
  --max-model-len <CTX_LEN> \
  --max-seq-len-to-capture <PREFILL_SEQ_LEN> \
  --max-num-seqs <MAX_NUM_SEQS> \
  --quantization mxfp6 \
  --kv-cache-dtype mxint8 \
  --override-qaic-config "num_cores:[num_cores] qpc_path:[qpc_path] ccl_enabled:True comp_ctx_lengths_prefill=[ccl_prefill_values] comp_ctx_lengths_decode=[ccl_decode_values]"

Run Inference¶

Once the server is running, send a request to the OpenAI-compatible endpoint:

curl http://localhost:<PORT>/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "Qwen/Qwen3-30B-A3B-Instruct-2507",
    "messages": [
      {"role": "user", "content": "<PROMPT>"}
    ],
    "max_tokens": 200
  }'

Useful Links¶

Environment Setup