Dynamic range: definition · Chip Field Guide

Beginner

The span from the smallest to the largest number a format can store.

Novice

The ratio between the largest and smallest non-zero values a format can represent, often counted in powers of two (binades). FP32 and BF16 cover about $2^{-126}$ to $2^{128}$ ; FP16 only about $2^{-14}$ to $2^{16}$ (down to $2^{-24}$ with subnormals).

Expert

Usually quoted in binades (octaves) from the smallest subnormal to the largest normal: 18 for OCP E4M3 and 32 for E5M2. It decides whether a tensor fits without a scale factor; a scale only shifts the window, it doesn’t widen it.