PTQ vs QAT: post-training quantization and quantization-aware training

Post-training quantization (PTQ) converts an already-trained model to lower precision using at most a small calibration sample and no retraining. Quantization-aware training (QAT) simulates quantization during training or fine-tuning so the weights learn to tolerate rounding error. PTQ is cheap and usually enough at 8 bits; QAT costs a training run but recovers accuracy when you push to 4 bits or below, or when PTQ alone falls short.

The difference at a glance

Both approaches end with the same artefact: a model whose weights, and sometimes activations, are stored as low-bit integers or small floats with scale factors. They differ in when the model learns about quantization. In PTQ, it never does; you pick good scales and rounding after the fact. In QAT, the forward pass rounds values during training, so gradient descent pushes the weights toward values that survive rounding.

Qualcomm AI Research's white paper on neural network quantization (Nagel et al., 2021) frames it this way: PTQ is a push-button step that needs neither retraining nor labelled data and is often sufficient for 8-bit, while QAT needs fine-tuning and training data but reaches lower bit widths with competitive accuracy.

Scales, zero-points and granularity

Both workflows share the same quantizer. Uniform affine (asymmetric) quantization maps a real value x to a b-bit integer using a scale s and zero-point z:

For a calibrated range [min, max]: s = (max − min) / (2^b − 1) and z = round(−min / s). Symmetric quantization fixes z = 0 and uses s = max|x| / (2^b−1 − 1) with a signed integer grid.

Symmetric is cheaper at run time, because there is no zero-point term to correct for when accumulating integer products. Asymmetric uses the full grid when data is lopsided, such as post-ReLU activations. The white paper recommends symmetric quantizers for weights and asymmetric ones for activations on typical accelerators.

Per-tensor, per-channel, per-group

Granularity is how many values share one scale. Per-tensor uses one scale for the whole tensor: simplest, but a single outlier ruins it. If one weight is 100 and the rest sit within ±1, a symmetric INT8 scale is 100 / 127 ≈ 0.79, so every weight smaller than about 0.39 in magnitude rounds to zero. Per-channel gives each output channel (row of a weight matrix) its own scale and is the standard for INT8 weights. Per-group (or block-wise) splits each row into groups, commonly 32, 64 or 128 weights, which is what 4-bit LLM methods such as GPTQ and AWQ rely on. The cost is metadata: a 16-bit scale per 128 weights adds 16 ÷ 128 = 0.125 bits per weight. Activations are usually per-tensor or, for LLMs, per-token.

Calibration and static vs dynamic quantization

Weights are known ahead of time, so their scales can be computed directly. Activations depend on the input, so you either measure them in advance (static) or compute them on the fly (dynamic).

Static quantization and calibration methods

Static PTQ runs a few hundred representative inputs through the model with observers attached, records activation statistics, and freezes a scale per activation tensor. Google's LiteRT (formerly TensorFlow Lite) documentation suggests roughly 100–500 samples for this representative dataset. The question is where to clip:

• Min-max: use the observed extremes. No clipping error, but outliers inflate the step size for everything else.
• Percentile: clip at, say, the 99.99th percentile of absolute values. Rare outliers saturate; typical values get finer resolution.
• MSE: search for the range that minimises mean squared error between original and quantized values, balancing rounding error against clipping error.
• Entropy (KL divergence): choose the threshold that minimises the KL divergence between the original and quantized activation histograms. NVIDIA's 2020 study of integer inference notes this is TensorRT's default.

That NVIDIA study (Wu et al.) compared max, entropy and percentile calibration and found that no single method was best for every network, so it recommends trying several. Its full 8-bit workflow (calibration, then partial quantization, then QAT only if needed) stayed within 1% of the floating-point baseline on every network it examined.

Dynamic quantization

Dynamic quantization stores weights in INT8 ahead of time but computes activation scales at run time from each input's actual range. There is no calibration step and no risk of a stale range, at the cost of computing min and max during inference. PyTorch's original guidance recommended dynamic quantization for LSTMs and Transformers, whose run time is dominated by loading weights, and static quantization for CNNs, where activation bandwidth matters. Modern LLM serving uses the same idea: FP8 and INT8 schemes with per-token dynamic activation scales need no calibration data.

How QAT works: fake quantization and the STE

QAT inserts fake quantization (quantize-then-dequantize) operations on weights and, optionally, activations. In the forward pass a value is scaled, rounded, clamped and immediately scaled back, so the tensor is still floating point but carries exactly the error the deployed integer model will see:

The problem is the backward pass. Rounding is a staircase whose gradient is zero almost everywhere and undefined at the steps, so naive backpropagation would never update anything. The straight-through estimator (STE), popularised by Bengio, Léonard and Courville (2013), simply treats rounding as the identity when computing gradients:

Gradients pass through unchanged inside the representable range and are blocked where the value is clipped.

The optimiser keeps full-precision "shadow" weights and updates them with these gradients; only the forward pass sees quantized values. After training you drop the fake-quant nodes and convert to real integer kernels. Variants learn the scale itself as a parameter, and for CNNs batch normalisation is usually folded into the preceding convolution before QAT so the simulated arithmetic matches deployment.

When QAT is worth it, and where QLoRA fits

Start with PTQ. Move to QAT when any of these apply:

• You are going below 8 bits for weights and activations, or below 4 bits for weights. Rounding error grows quickly as the grid coarsens, and PTQ has no way to let the network compensate.
• The model is small or already efficiency-tuned. Compact vision networks have little redundancy to absorb error.
• PTQ with good calibration and per-channel scales still misses your accuracy target. Wu et al. suggest first leaving the most sensitive layers in floating point, then fine-tuning with QAT for roughly 10% of the original schedule, starting at 1% of the original learning rate.
• You ship one checkpoint to many users, so a one-time training cost is amortised. Google's Gemma 3 QAT release is an example: about 5,000 QAT steps using the full-precision model's output probabilities as targets, which Google reports cut the perplexity drop at Q4_0 by 54%.

QAT does not pay off when PTQ is already within noise of the baseline, as is common for INT8 and FP8 on large models.

QLoRA: PTQ base, trained adapters

QLoRA (Dettmers et al., 2023) is a hybrid. The base model is quantized post-training to NF4 and frozen; small low-rank adapters are trained in BF16 on top. The adapters do learn in the presence of quantization error, but the quantized weights themselves never change, so it is not QAT in the strict sense. Its goal is cheap fine-tuning, not a better quantized model. torchao and Unsloth now also offer QAT combined with LoRA, which brings the two ideas closer.

PTQ and QAT in PyTorch and TensorFlow

PyTorch: torchao

New quantization work in the PyTorch ecosystem happens in torchao, whose quantize_ function swaps nn.Linear layers according to a config object. Dynamic INT8 PTQ (per-token activations, per-channel weights) and the two-step QAT flow look like this, following the current torchao documentation:

from torchao.quantization import (
    quantize_, Int8DynamicActivationInt8WeightConfig, Int4WeightOnlyConfig,
)
from torchao.quantization.qat import QATConfig

# PTQ: dynamic INT8 activations + INT8 weights, no calibration
quantize_(model, Int8DynamicActivationInt8WeightConfig())

# QAT (on a fresh float model): insert fake-quantized linears, train, convert
base_config = Int4WeightOnlyConfig(group_size=32)
quantize_(model, QATConfig(base_config, step="prepare"))
train_loop(model)  # your usual fine-tuning loop
quantize_(model, QATConfig(base_config, step="convert"))

At the time of writing, the docs list Int4WeightOnlyConfig as the PTQ base config supported by QATConfig; finer control is available through fake-quantize configs such as IntxFakeQuantizeConfig.

TensorFlow Model Optimization

The Keras QAT API wraps a model with fake-quant nodes; the result is "quantization aware" but still float32 until the LiteRT converter produces INT8 weights:

import tensorflow as tf
import tensorflow_model_optimization as tfmot

qat_model = tfmot.quantization.keras.quantize_model(model)
qat_model.compile(optimizer="adam", loss=loss_fn, metrics=["accuracy"])
qat_model.fit(train_ds, epochs=1)

converter = tf.lite.TFLiteConverter.from_keras_model(qat_model)
converter.optimizations = [tf.lite.Optimize.DEFAULT]
tflite_model = converter.convert()

The toolkit supports Sequential and Functional Keras models, and its deployment path supports 8-bit only. The official example runs on the legacy tf_keras package, so check Keras version compatibility before adopting it. For PTQ, the converter offers dynamic-range, full-integer (with a representative dataset) and float16 modes.