Quantization cheat sheet
Every formula and rule of thumb from the site on one page. Print it or save it as a PDF from your browser.
Core quantization
x̂ = s · (q − z)
Symmetric: s = max|x| ÷ (2b−1 − 1), z = 0. Asymmetric: s = (max − min) ÷ (2b − 1), z = round(−min ÷ s).
LLM memory
Add about 1 GB plus 5% of weights for runtime overhead. Llama 3.1 8B: 128 KiB of FP16 KV cache per token, so 1 GiB per 8K tokens.
| Format | ≈ bits/weight | 8B model | 70B model |
|---|---|---|---|
| BF16 | 16 | 15.0 GB | 131 GB |
| Q8_0 / FP8 | 8–8.5 | 7.5–8.0 GB | 66–70 GB |
| Q6_K | 6.56 | 6.1 GB | 54 GB |
| Q5_K_M | 5.70 | 5.3 GB | 47 GB |
| Q4_K_M | 4.89 | 4.6 GB | 40 GB |
| GPTQ/AWQ 4-bit | ≈4.25 | 4.0 GB | 35 GB |
| Q3_K_M | 4.00 | 3.7 GB | 33 GB |
| Q2_K | 3.16 | 3.0 GB | 26 GB |
Sizes in GiB, weights only, computed from 8.03B and 70.6B parameters. Effective GGUF bits per weight from the llama.cpp quantize README.
Choosing a format, fast
- Near-lossless: 8-bit (Q8_0, FP8, INT8 W8A8).
- Best balance for local use: Q4_K_M to Q6_K.
- GPU serving at scale: FP8 on newer GPUs; AWQ or GPTQ 4-bit when memory is the limit.
- Below about 3 bits per weight, quality drops faster; use importance-matrix (imatrix) i-quants and test carefully.
Signals and ADCs
Each doubling of sample rate adds ≈ 3 dB (½ bit) without noise shaping.
Audio
| Bit depth | Theoretical dynamic range |
|---|---|
| 16-bit | ≈ 96 dB |
| 24-bit | ≈ 144 dB |
| 32-bit float | huge headroom; practical limit set by analog stages |
Dither once, at the final word-length reduction (e.g. 24 → 16-bit). TPDF dither spans ±1 LSB.
Music timing
Swing (drum-machine convention): 50% straight, ≈ 66.7% triplet. Second note of each pair sits at swing × pair length.
Physics
h = 6.62607015 × 10⁻³⁴ J·s (exact). Balmer lines: 656, 486, 434, 410 nm.
One clear explainer a week
New tools, guides and formats across AI, signals, audio and physics. No spam; unsubscribe any time.