The quantization formats atlas
32 formats and methods in one sortable table: effective bits per weight, what needs calibration, where each one runs and the primary source behind it.
| Format | Bits / weight | Type | Hardware and runtimes | How it works | Source |
|---|---|---|---|---|---|
| GGUF Q8_0 GGUF legacy | 8.5 | weight-only Calibration: none | CPU, Apple Silicon, NVIDIA, AMD llama.cpp, Ollama, LM Studio | Weights are split into blocks of 32, and each block stores 32 signed 8-bit integers plus one FP16 scale. A weight is rebuilt by multiplying its integer by the block scale; llama.cpp ignores an importance matrix for this type. Best for: A near-lossless reference copy at roughly half the size of FP16. Bits: approx.; 8.5008 measured on Llama-3.1-8B (llama.cpp quantize README); nominal 8.5 = 32 int8 values + one FP16 scale per block | llama.cpp tools/quantize README (bits/weight table) |
| GGUF Q6_K GGUF k-quant | 6.56 | weight-only Calibration: optional imatrix | CPU, Apple Silicon, NVIDIA, AMD llama.cpp, Ollama, LM Studio | Weights are grouped into 256-weight super-blocks made of 16 blocks of 16, with a 6-bit value per weight and an 8-bit scale per block. One FP16 scale per super-block sets the overall range. Best for: Quality-first local inference that still saves about a fifth of the memory of Q8_0. Bits: approx.; 6.5633 measured on Llama-3.1-8B; nominal 6.5625 for the Q6_K block itself | llama.cpp tools/quantize README (bits/weight table) |
| GGUF Q5_K_M GGUF k-quant | 5.7 | weight-only Calibration: optional imatrix | CPU, Apple Silicon, NVIDIA, AMD llama.cpp, Ollama, LM Studio | Most tensors use Q5_K: 5-bit values with 6-bit scales and minimums for each 32-weight block inside 256-weight super-blocks. llama.cpp raises selected attention-value and feed-forward-down layers to Q6_K to protect the most sensitive weights. Best for: A high-quality everyday choice when you have a little more memory than Q4_K_M needs. Bits: approx.; 5.7036 measured on Llama-3.1-8B, whole-file average of a mix (Q5_K nominal 5.5) | llama.cpp tools/quantize README (bits/weight table) |
| GGUF Q5_K_S GGUF k-quant | 5.57 | weight-only Calibration: optional imatrix | CPU, Apple Silicon, NVIDIA, AMD llama.cpp, Ollama, LM Studio | Applies Q5_K to almost every tensor and skips the extra Q6_K upgrades that the M variant adds. The output layer still gets a higher-precision type. Best for: Slightly smaller than Q5_K_M, at a small, measurable cost in quality. Bits: approx.; 5.5704 measured on Llama-3.1-8B (Q5_K nominal 5.5) | llama.cpp tools/quantize README (bits/weight table) |
| GGUF Q4_K_M GGUF k-quant | 4.89 | weight-only Calibration: optional imatrix | CPU, Apple Silicon, NVIDIA, AMD llama.cpp, Ollama, LM Studio | Most tensors use Q4_K: 4-bit values with 6-bit scales and minimums for each 32-weight block in a 256-weight super-block. About half of the attention-value and feed-forward-down layers, chosen by a fixed layer pattern, are stored as Q6_K instead. Best for: The common default for local LLMs, with good quality at about 30% of FP16 size. Bits: approx.; 4.8944 measured on Llama-3.1-8B, whole-file average of a mix (Q4_K nominal 4.5) | llama.cpp tools/quantize README (bits/weight table) |
| GGUF Q4_K_S GGUF k-quant | 4.67 | weight-only Calibration: optional imatrix | CPU, Apple Silicon, NVIDIA, AMD llama.cpp, Ollama, LM Studio | Uses Q4_K for nearly all tensors and upgrades only a few early attention-value and feed-forward-down layers to Q5_K. It is smaller than Q4_K_M because it skips most of the Q6_K upgrades. Best for: Tighter memory budgets where Q4_K_M does not quite fit. Bits: approx.; 4.6672 measured on Llama-3.1-8B (Q4_K nominal 4.5) | llama.cpp tools/quantize README (bits/weight table) |
| GGUF Q4_0 GGUF legacy | 4.5 | weight-only Calibration: optional imatrix | CPU, Apple Silicon, NVIDIA, AMD llama.cpp, Ollama, LM Studio | A symmetric round-to-nearest scheme. Each block of 32 weights stores 4-bit integers and one FP16 scale, with no offset, and a weight is rebuilt as integer × scale. Best for: Maximum compatibility and simple kernels; for quality, Q4_K_M or IQ4_XS are usually better. Bits: nominal: 32 4-bit values + one FP16 scale = 18 bytes per 32 weights; whole-file average runs slightly higher | Hugging Face Hub docs: GGUF quantization types |
| GGUF Q3_K_M GGUF k-quant | 4 | weight-only Calibration: optional imatrix | CPU, Apple Silicon, NVIDIA, AMD llama.cpp, Ollama, LM Studio | Most tensors use Q3_K, which stores 3-bit values in 16-weight blocks with 6-bit scales. Attention-value, attention-output and feed-forward-down tensors are raised to Q4_K or Q5_K, which pushes the file average close to 4 bits. Best for: Fitting a larger model into limited RAM or VRAM when a 4-bit file will not fit. Bits: approx.; 3.9960 measured on Llama-3.1-8B, whole-file average of a mix (Q3_K nominal 3.4375) | llama.cpp tools/quantize README (bits/weight table) |
| GGUF Q3_K_S GGUF k-quant | 3.64 | weight-only Calibration: optional imatrix | CPU, Apple Silicon, NVIDIA, AMD llama.cpp, Ollama, LM Studio | Applies Q3_K to nearly every tensor and skips the higher-precision upgrades that the M and L variants use. Only special tensors such as the output layer keep a higher-precision type. Best for: Very tight memory before you drop to 2-bit, with a clearly measurable quality loss. Bits: approx.; 3.6429 measured on Llama-3.1-8B (Q3_K nominal 3.4375) | llama.cpp tools/quantize README (bits/weight table) |
| GGUF Q2_K GGUF k-quant | 3.16 | weight-only Calibration: optional imatrix | CPU, Apple Silicon, NVIDIA, AMD llama.cpp, Ollama, LM Studio | Most weights use Q2_K, which stores 2-bit values with 4-bit scales and minimums per 16-weight block. Attention-value, attention-output and feed-forward-down tensors are kept at Q3_K or Q4_K, which is why the file averages more than 3 bits per weight. Best for: Running a very large model on small hardware when a noticeable quality drop is acceptable. Bits: approx.; 3.1593 measured on Llama-3.1-8B, whole-file average of a mix (Q2_K block nominal 2.625) | llama.cpp tools/quantize README (bits/weight table) |
| GGUF IQ4_XS GGUF i-quant | 4.46 | weight-only Calibration: optional imatrix | CPU, Apple Silicon, NVIDIA, AMD llama.cpp, Ollama, LM Studio | Each weight is a 4-bit index into a fixed, non-uniform table of 16 values, so the grid is denser near zero where most weights sit. Weights are packed in 256-weight super-blocks with compact sub-block scales, and an importance matrix improves results but is not required. Best for: 4-bit-class quality in less memory than Q4_K_S. Bits: approx.; 4.4597 measured on Llama-3.1-8B; nominal 4.25 for the IQ4_XS block | llama.cpp tools/quantize README (bits/weight table) |
| GGUF IQ3_XXS GGUF i-quant | 3.25 | weight-only Calibration: required | CPU, Apple Silicon, NVIDIA, AMD llama.cpp, Ollama, LM Studio | Small groups of weights are stored as indices into a fixed 256-entry codebook grid instead of as independent values. llama.cpp refuses to produce this type without an importance matrix (imatrix). Best for: The smallest usable 3-bit-class file when you can build an importance matrix first. Bits: approx.; 3.2548 measured on Llama-3.1-8B; nominal 3.06 for the IQ3_XXS block | llama.cpp tools/quantize README (bits/weight table) |
| GGUF IQ2_XS GGUF i-quant | 2.59 | weight-only Calibration: required | CPU, Apple Silicon, NVIDIA, AMD llama.cpp, Ollama, LM Studio | Groups of eight weights are encoded as an index into a 512-entry codebook of 8-value patterns, plus sign bits and small block scales. llama.cpp requires an importance matrix for this type. Best for: Fitting very large models into consumer memory, accepting a large quality loss. Bits: approx.; 2.5882 measured on Llama-3.1-8B; nominal 2.31 for the IQ2_XS block | llama.cpp tools/quantize README (bits/weight table) |
| GGUF IQ1_S GGUF i-quant | 2 | weight-only Calibration: required | CPU, Apple Silicon, NVIDIA, AMD llama.cpp, Ollama, LM Studio | Groups of weights are mapped to entries in a large codebook of patterns built from −1, 0 and +1, with a shared scale per block. llama.cpp requires an importance matrix, and on Llama-3-8B its own perplexity table shows a very large accuracy loss. Best for: Experiments and extreme memory limits only; on most models quality collapses. Bits: approx.; 2.0042 measured on Llama-3.1-8B whole file; nominal 1.56 for the IQ1_S block (other tensors stay at higher precision) | llama.cpp tools/quantize README (bits/weight table) |
| GPTQ 4-bit (group size 128) GPTQ | 4.16 | weight-only Calibration: required | NVIDIA, AMD, CPU Transformers (GPTQModel), vLLM, TensorRT-LLM | Quantizes one layer at a time, using second-order (Hessian) information from a small calibration set to adjust the remaining weights so each rounding error is compensated. The paper reports quantizing a 175B model in about four GPU hours. Best for: Fast 4-bit GPU serving of models that already ship GPTQ checkpoints. Bits: approx., computed: 4-bit weights + one FP16 scale and one packed 4-bit zero per 128-weight group; FP16 embeddings and head add more | GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers (arXiv:2210.17323) |
| AWQ 4-bit AWQ | 4.16 | weight-only Calibration: required | NVIDIA, AMD, CPU Transformers, vLLM, TensorRT-LLM | Uses activation statistics from calibration data to find the roughly 1% of weight channels that matter most, then scales those channels up before quantizing so they lose less precision. It needs no backpropagation; the original AutoAWQ library is archived and the method now lives in vLLM's llm-compressor. Best for: 4-bit GPU inference that generalizes well across domains, including instruction-tuned and multimodal models. Bits: approx., computed for the common group size 128: 4-bit weights + FP16 scale and packed 4-bit zero per group | AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration (arXiv:2306.00978) |
| EXL2 (variable bpw) EXL2 | varies | weight-only Calibration: required | NVIDIA ExLlamaV2, TabbyAPI, text-generation-webui | Mixes 2, 3, 4, 5, 6 and 8-bit quantization within and across layers, using GPTQ's optimization method and calibration error measurements to pick the combination that meets a target average bitrate. The ExLlamaV2 repo is now archived in favour of ExLlamaV3 and its EXL3 format. Best for: Hitting an exact VRAM target on a single consumer NVIDIA GPU. Bits: variable: any average from 2 to 8 bpw, chosen at conversion time | ExLlamaV2 GitHub repository (EXL2 format) |
| bitsandbytes LLM.int8() bitsandbytes | 8 | weight+activation Calibration: none | NVIDIA, AMD, CPU, Apple Silicon Transformers, bitsandbytes | Matrix multiplications run in 8-bit with separate scaling for each row and column, while the rare outlier feature dimensions are pulled out and multiplied in 16-bit. The paper reports that more than 99.9% of values are still multiplied in 8-bit. Best for: Loading a model in about half its FP16 memory with no calibration step. Bits: 8-bit weights; outlier feature columns are computed in 16-bit | LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale (arXiv:2208.07339) |
| bitsandbytes NF4 bitsandbytes | 4.13 | weight-only Calibration: none | NVIDIA, AMD, CPU, Apple Silicon Transformers, bitsandbytes, PEFT | Each weight is snapped to one of 16 levels placed at quantiles of a normal distribution, scaled by the absolute maximum of its 64-weight block, and the levels include an exact zero. Double quantization then compresses the per-block scales themselves. Best for: QLoRA fine-tuning and quick 4-bit loading of Hugging Face models without calibration. Bits: approx.; 4 + 0.127 bits of constants with double quantization at block size 64 (4.5 without), per the QLoRA paper | QLoRA: Efficient Finetuning of Quantized LLMs (arXiv:2305.14314) |
| HQQ (Half-Quadratic Quantization) HQQ | varies | weight-only Calibration: none | NVIDIA, CPU Transformers, vLLM, HQQ library | Leaves the weights fixed and solves only for each group's zero-point and scale, minimizing a sparsity-promoting error (an lp norm with p below 1) with a fast closed-form half-quadratic solver. The authors report processing Llama-2-70B in under five minutes. Best for: Quantizing very large models in minutes when you have no calibration data. Bits: variable: 8, 4, 3, 2 or 1-bit weights plus per-group scale and zero-point; depends on group size | Half-Quadratic Quantization of Large Machine Learning Models (Dropbox HQQ blog) |
| SmoothQuant W8A8 (INT8) Integer PTQ | 8 | weight+activation Calibration: required | NVIDIA, CPU vLLM, llm-compressor | Activation outliers make activations hard to quantize, so SmoothQuant divides each activation channel by a smoothing factor and multiplies the matching weight row by the same factor, moving difficulty from activations to weights without changing the layer's output. The factors come from calibration statistics; the paper used 512 sentences and a default migration strength of 0.5. Best for: Running weights and activations in INT8 on hardware with fast integer matrix units. Bits: INT8 weights and INT8 activations | SmoothQuant: Accurate and Efficient Post-Training Quantization for LLMs (arXiv:2211.10438) |
| FP8 E4M3 Floating point | 8 | weight+activation Calibration: none | NVIDIA, AMD vLLM, TensorRT-LLM, Transformer Engine | An 8-bit float with more precision and less range than E5M2. Its largest normal value is 448 and it has no infinity encoding, and the FP8 paper recommends it for weights and activations; dynamic per-tensor scaling needs no calibration data. Best for: Halving memory versus BF16 on GPUs with native FP8 math (NVIDIA Ada, Hopper, Blackwell). Bits: 1 sign, 4 exponent, 3 mantissa bits; per-tensor or finer scales add a little | FP8 Formats for Deep Learning (arXiv:2209.05433) |
| FP8 E5M2 Floating point | 8 | training/inference Calibration: none | NVIDIA Transformer Engine, vLLM | An 8-bit float that trades a mantissa bit for an extra exponent bit, reaching a maximum of 57,344 and following IEEE conventions for infinities and NaN. The FP8 paper recommends it for gradient tensors, and Transformer Engine's hybrid recipe uses it in the backward pass. Best for: Gradients in FP8 training; it also exists as a KV-cache dtype option in vLLM. Bits: 1 sign, 5 exponent, 2 mantissa bits | FP8 Formats for Deep Learning (arXiv:2209.05433) |
| MXFP4 Microscaling | 4.25 | weight-only Calibration: none | NVIDIA, AMD, CPU, Apple Silicon llama.cpp, Ollama, LM Studio, vLLM, TensorRT-LLM, Transformers | Each block of 32 values shares one power-of-two scale (E8M0), and each element is a tiny 4-bit float (E2M1). The spec covers activations and training too, but the best-known public use is storing gpt-oss's mixture-of-experts weights. Best for: Open-weight models released natively in MXFP4, such as gpt-oss MoE weights. Bits: computed from the OCP MX v1.0 spec: 4-bit E2M1 elements + one 8-bit E8M0 scale per 32-element block | OCP Microscaling Formats (MX) Specification v1.0 |
| MXFP8 Microscaling | 8.25 | training/inference Calibration: none | NVIDIA Transformer Engine, llm-compressor | Gives every block of 32 FP8 values its own power-of-two scale, so one outlier only affects its own block. That lets E4M3 be used for all tensors in training rather than switching to E5M2 for gradients. Best for: FP8 training and inference on NVIDIA Blackwell with finer-grained scaling than per-tensor FP8. Bits: computed from the OCP MX v1.0 spec: 8-bit FP8 (E4M3 or E5M2) elements + one 8-bit E8M0 scale per 32-element block | OCP Microscaling Formats (MX) Specification v1.0 |
| NVFP4 Microscaling | 4.5 | weight+activation Calibration: required | NVIDIA TensorRT-LLM, vLLM, TensorRT Model Optimizer, llm-compressor | Like MXFP4 it stores 4-bit E2M1 values, but it uses smaller 16-value blocks and an FP8 E4M3 block scale that can take non-power-of-two values, plus a per-tensor FP32 scale. For W4A4, a small calibration sample sets the per-tensor activation scale. Best for: 4-bit inference on NVIDIA Blackwell GPUs with accuracy close to FP8. Bits: approx.; 4-bit E2M1 value + one FP8 E4M3 scale per 16 values, plus one FP32 scale per tensor | Introducing NVFP4 for Efficient and Accurate Low-Precision Inference (NVIDIA Technical Blog) |
| BF16 / FP16 (16-bit baseline) Floating point | 16 | training/inference Calibration: none | CPU, Apple Silicon, NVIDIA, AMD llama.cpp, vLLM, Transformers | Most models are released in one of these 16-bit floats. BF16 keeps FP32's exponent range with less precision, while FP16 has more precision but a much narrower range. Best for: The unquantized reference that every other format is measured against. Bits: BF16: 1 sign, 8 exponent, 7 mantissa bits; FP16: 1 sign, 5 exponent, 10 mantissa bits | BFloat16: The secret to high performance on Cloud TPUs (Google Cloud Blog) |
| BitNet b1.58 (ternary) Research | 1.58 | weight+activation Calibration: QAT | CPU, NVIDIA bitnet.cpp, llama.cpp | Every weight is −1, 0 or +1 after scaling by the matrix's mean absolute value, and activations use 8 bits per token. The model is trained this way rather than converted afterwards, and the paper reports it matching an FP16 LLaMA baseline in perplexity from 3B parameters. Best for: Models trained from the start for ternary weights that run efficiently on CPUs. Bits: log2(3) ≈ 1.58 bits of information per ternary weight; packed storage varies (llama.cpp TQ1_0 1.69, TQ2_0 2.06 bpw) | The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits (arXiv:2402.17764) |
| AQLM Research | varies | weight-only Calibration: required | NVIDIA, CPU Transformers, vLLM | Represents each small group of weights as the sum of entries from several learned codebooks (additive quantization), with the codebooks tuned jointly across each transformer block using calibration data. The authors report it is Pareto-optimal for accuracy versus size below 3 bits per parameter. Best for: Extreme 2-bit compression where accuracy per byte matters most. Bits: variable by codebook scheme; typical checkpoints are labelled 2-bit (e.g. 1x16, 2x8), and 1-bit variants exist | Extreme Compression of Large Language Models via Additive Quantization (arXiv:2401.06118) |
| QuIP# Research | varies | weight-only Calibration: required | NVIDIA quip-sharp (reference code) | Applies a randomized Hadamard transform so weights look evenly spread (incoherence processing), then vector-quantizes groups of eight using a codebook built on the E8 lattice, with optional fine-tuning. Its authors have since moved on to QTIP, and the repo is no longer actively developed. Best for: Research-grade quality at 2 to 4 bits per weight. Bits: 2, 3 or 4 bpw configurations | QuIP#: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebooks (arXiv:2402.04396) |
| KV cache Q8_0 (llama.cpp) KV cache | 8.5 | KV cache Calibration: none | CPU, Apple Silicon, NVIDIA, AMD llama.cpp, Ollama | Stores attention keys and values in 8-bit blocks instead of FP16, roughly halving cache memory. In llama.cpp a quantized V cache requires Flash Attention; other allowed cache types include q4_0, q4_1, q5_0, q5_1 and iq4_nl. Best for: Fitting longer context into the same memory with very little quality impact. Bits: nominal; same Q8_0 block layout (32 int8 values + FP16 scale) applied to cached keys and values | llama.cpp server README (--cache-type-k / --cache-type-v) |
| FP8 KV cache (vLLM) KV cache | 8 | KV cache Calibration: none | NVIDIA, AMD vLLM, TensorRT-LLM | Keys and values are written to the cache in FP8 instead of 16-bit. Without calibration every scale defaults to 1.0, and vLLM recommends calibrating scales with llm-compressor for best accuracy. Best for: Serving more tokens or longer contexts per GPU in production inference servers. Bits: FP8 E4M3 or E5M2 values; scales per tensor or per attention head | vLLM docs: Quantized KV Cache |
Bits per weight are approximate effective values including scales and, for GGUF k-quants, the higher-precision tensors the mix keeps. Real file sizes vary by architecture. Plug any value into the VRAM calculator to size a model.
Want this done for your model?
Tell us your model, hardware and target. We send back a free optimization plan: recommended format, expected memory and a quality-check method.