Fundamentals / October 2, 2026 / 3 min read
Quantization in Pictures: Fewer Bits, Fewer Colours
A photo stores red, green and blue as numbers. Fewer bits give fewer colours, and rounding each pixel alone makes the damage worse. The demo shows the fix.
Fewer bits per number, fewer colours per pixel
Numbers
- PSNR
- —
- SSIM
- —
- Shape error
- —
- Detail error
- —
PSNR scores the average pixel error. SSIM scores the structure. Shape is the error that remains after a blur; detail is what the blur removes. Higher is better for all four.
Numbers
- PSNR
- —
- SSIM
- —
- Shape error
- —
- Detail error
- —
PSNR scores the average pixel error. SSIM scores the structure. Shape is the error that remains after a blur; detail is what the blur removes. Higher is better for all four.
The error map shows where the quantiser moved the picture: brighter, darker. Move the pointer over a panel for a magnifier. Open Numbers under a photo for the measurements.
What this maps to in model weights
A quantiser stores each weight on a fixed ladder: fewer bits per weight means fewer values that weight can take, the same trade as fewer colours per pixel.Round to nearest uses one scale for the whole tensor and rounds each weight on its own. Per-tensor int8 works this way, and every other scheme starts here.
Error passed on is what most real quantisers do. GPTQ rounds weights one at a time, then changes the weights that are not quantised yet to take the error. Stochastic rounding spreads the error at random. The pictures show the trade: the bands go, and grain replaces them. The eye reads the shape; the per-pixel scores punish the grain.
One limit of the analogy: a quantiser picks its levels from the weights that it will store, so it can put the ladder where the values are. A camera has already fixed each pixel at 8 bits, so the demo can only use a thinner ladder. The per-block scales and the mixed-precision tricks are not in these pictures.
| Demo | Weight-quantisation analogue | Where it is used |
|---|---|---|
| Round to nearest | one scale for the tensor, evenly spaced levels, every weight rounded on its own | per-tensor int8, and the intuition behind every other scheme |
| Error passed on | error feedback: the weights still to be quantised absorb the error of the ones already done | GPTQ and most post-training quantisers; stochastic rounding spreads it randomly |
Fewer colours, visibly
A photo stores red, green and blue as three numbers per pixel. Eight bits per number gives 16.7 million colours, so the picture looks like the original. Round to nearest keeps eight values per channel at three bits, and four at two bits. That is 512 colours, then 64. Every pixel is rounded on its own, and the ladder never changes.
A sky is a smooth change from one tone to the next. When the ladder has few values, that gentle change disappears: whole areas round to the same value, and the sky breaks into flat bands. Hard edges keep their shape. The soft gradient is the part that suffers.
The error is a choice
Round to nearest throws the error away. Error passed on keeps the same ladder and the same colours, and gives each pixel’s error to the neighbours that are not rounded yet. A pixel that should be 127 can only be 85 or 170, so it takes a mix of the two. The errors cancel out, and from a normal distance the eye sees the average.
The bands disappear, and fine grain takes their place. The picture looks right from a distance, while the per-pixel scores get worse. That is the trade: the eye reads the shape, and the scores punish the grain. The numbers under each photo show it.
GPTQ does the same with weights. It rounds one weight at a time, then changes the weights that are not quantised yet, so they carry the error. Stochastic rounding spreads the error at random instead.
What this means for language models
Both pictures use one ladder for the whole image. Per-tensor int8 does the same with weights: one scale for the tensor, and every weight rounds on its own.
The formats that you download go further. Q4_K_M stores a scale and a minimum for each block of weights, so the ladder fits the values in that block. UD quants, the i-quants and EXL3 keep their precision for the weights that matter most and give fewer bits to the rest. They find those weights by sensitivity: how much the output moves when a weight changes. The demo cannot do that — the camera has already fixed each pixel at eight bits.
Two limits: these are colours, not weights, and the demo shows the two pixel methods only. The quantization guide covers the rest, and its advice does not change: Q4_K_M is the safe default.
The KV cache shows the same lesson.
The workstation photo is the dual RTX 3090 build. The sky photo is Pexels 38459847.