Memory bandwidth
Beginner
How fast numbers can be brought from memory to the part of the chip that does math. Think of it as the width of a pipe.
Novice
The rate at which data moves between a chip and its memory, in bytes per second. AI chips with stacked memory reach terabytes per second.
Expert
Quoted as a theoretical peak (bus width × data rate). Sustained bandwidth is lower, and the roofline paper measures it with microbenchmarks rather than trusting the pin rate.
Explained in What the workload needs (Architectures).
See also: HBM (High Bandwidth Memory), DRAM.