[IlyasMoutawwakil]

Adds ModelOptOnnxQuantizer, a quantizer for the converted ONNX model built on NVIDIA's ModelOpt (modelopt.onnx.quantization.quantize), for TensorRT (e.g. ONNX Runtime's TensorRT execution provider).

from transformers.exporters import ModelOptOnnxConfig, ModelOptOnnxQuantizer, OnnxConfig, OnnxExporter

quantizer = ModelOptOnnxQuantizer(ModelOptOnnxConfig(disable_mha_qdq=True), calibration_dataset=samples)
exported = OnnxExporter().export(model.float(), inputs, OnnxConfig(quantizer=quantizer))

ModelOpt runs ONNX Runtime's static quantization with TensorRT-guided node selection (Q/DQ placed where TensorRT fuses, per-channel weights, attention matmuls optionally left out), then converts what it leaves unquantized to fp16. Unlike OnnxRuntimeQuantizer, the result is laid out for TensorRT's explicit quantization.

• ModelOptOnnxConfig: the main arguments of quantize (quantize_mode, calibration_method, calibration_eps, op/node selection, high_precision_dtype, disable_mha_qdq, ...); unset fields keep ModelOpt's defaults.
• The quantizer saves the model with external data, so models past 2GB work, and reads the calibration set through the same feed reader as OnnxRuntimeQuantizer, which now also implements get_first and rewind (ModelOpt reads the first feed for its MHA analysis).
• is_modelopt_available / require_modelopt, docs entries.

ModelOpt 0.48 pins onnx~=1.21 on Linux, hence the tested versions.

Tests

test_modelopt_onnx (dense, MoE): the tiny Llama/Mixtral decode component quantized with ModelOpt int8 (calibrated on CPU, kept in fp32), checked for QuantizeLinear nodes and finite outputs in ONNX Runtime.

RUN_SLOW=1 pytest tests/exporters/test_quantization.py -k "modelopt or onnxruntime_onnx"

🤖 Generated with Claude Code

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Pages changed in this PR:

• en/main_classes/exporters