Integer quantization (INT8, INT4)
Beginner
Storing numbers as small whole numbers, like −127 to 127, plus one shared multiplier.
Novice
Representing values as signed integers times a scale: with from −127 to 127 for INT8 or −7 to 7 for INT4. The steps are evenly spaced, so the format is precise for values near the top of the range and coarse for small ones relative to their size.
Expert
Symmetric () or affine (zero-point ) uniform quantizers. Products of INT8 operands are exact 16-bit integers and accumulate exactly in INT32, which makes the hardware small. Accuracy depends on granularity and calibration, and on handling outliers that stretch the range.
Explained in Number formats (Architectures).
See also: Quantization, Outliers (in activations).