Gpt oss 120b

Model Overview

OpenAI’s GPT-OSS models (gpt-oss-120b & gpt-oss-20b) are open-weight models designed for powerful reasoning, agentic tasks and versatile developer use cases. GPT-OSS-120B is used for production, general purpose, high reasoning use cases that fit into a single 80GB GPU.

• Model Architecture: 117B parameters with 5.1B active parameters. Trained on harmony response format and should only be used with the harmony format as it will not work correctly otherwise.
• Model Source: openai/gpt-oss-120b
• License: Apache 2.0 license. Build freely without copyleft restrictions or patent risk—ideal for experimentation, customization, and commercial deployment.
• Configurable reasoning effort: Easily adjust the reasoning effort (low, medium, high) based on your specific use case and latency needs.
• Full chain-of-thought: Gain complete access to the model’s reasoning process, facilitating easier debugging and increased trust in outputs. It’s not intended to be shown to end users.
• Fine-tunable: Fully customize models to your specific use case through parameter fine-tuning.
• Agentic capabilities: Use the models’ native capabilities for function calling, web browsing, Python code execution, and Structured Outputs.
• Native MXFP4 quantization: The models were post-trained with MXFP4 quantization of the MoE weights, making gpt-oss-120b run on a single 80GB GPU.

QPC Configurations

Precision	SoCs / Tensor slicing	NSP-Cores (per SoC)	Full Batch Size	Chunking Prompt Length	Context Length (CL)	CCL_Enabled	QPC URL	QPC Size	QPC Download	Onnx URL	Onnx Download	Generation Date
MXFP6	4	16	1	1	32768	False	https://qualcom-qpc-models.s3-accelerate.amazonaws.com/SDK1.21.6/openai/gpt-oss-120b/openai_gptoss_120b_qpc_16cores_1fb_32kcl_4soc_sdk_1_21_6.tar.gz	90GB	Download	https://qualcom-qpc-models.s3-accelerate.amazonaws.com/SDK1.21.6/openai/gpt-oss-120b/openai_gptoss_120b_onnx_sdk_1_21_6.tar.gz	Download	8-Jul-2026
MXFP6	4	16	1	1	32768	True	https://qualcom-qpc-models.s3-accelerate.amazonaws.com/SDK1.21.6/openai/gpt-oss-120b/openai-gpt-oss-120b_qpc_16cores_1pl_1024,2048,4096,8192,16384,32768ccl_4devices_mxfp6_mxint8_ccl.tar	90GB	Download	https://qualcom-qpc-models.s3-accelerate.amazonaws.com/SDK1.21.6/openai/gpt-oss-120b/openai-gpt-oss-120b_ONNX_16cores_1pl_1024,2048,4096,8192,16384,32768ccl_4devices_mxfp6_mxint8_ccl.tar	Download	13-Jul-2026

Run This Model

Download QPCs

mkdir -p openai/gpt-oss-120b
cd openai/gpt-oss-120b

# Download QPC
wget <QPC_Download_URL>
tar xzvf <qpc_filename.tar.gz>

Run QPC

export VLLM_QAIC_QPC_PATH=/path/to/qpc

python3 -m vllm.entrypoints.openai.api_server \
  --host 0.0.0.0 \
  --port <PORT> \
  --model openai/gpt-oss-120b  \
  --device-group <DEVICE_IDS> \
  --max-model-len <CTX_LEN> \
  --max-seq-len-to-capture <PREFILL_SEQ_LEN> \
  --max-num-seqs <MAX_NUM_SEQS> \
  --dtype bfloat16   \
  --kv-cache-dtype mxint8 \
  --model-impl auto

Useful Links

• Environment Setup