Peak throughput
Beginner
The fastest a chip could possibly do math, if every part of it were busy every moment. It’s the big number printed on the box.
Novice
The theoretical maximum rate of operations, usually in trillions of operations per second (TFLOP/s or TOPS). It is calculated, not measured: number of math units × operations per clock tick × clock rate.
Expert
Computed at the boost clock for one precision, often with structured sparsity doubling it. It assumes every multiply-accumulate unit issues every cycle, so it ignores memory stalls, data movement, non-matrix work and power or thermal throttling.
Explained in Comparing chips (Architectures).
See also: Utilization (of compute), FLOP and FLOP/s.