Memory wall
Beginner
The problem that a chip can do math much faster than it can fetch the numbers to do math on, so its math parts sit waiting.
Novice
The widening gap between how fast processors compute and how fast memory can deliver data. Wulf and McKee named it in 1995; in AI chips it shows up as math units idling while weights stream in from DRAM.
Expert
Compute throughput has grown faster than DRAM bandwidth, capacity and latency for decades (peak FLOPS about 3.0× per two years against 1.6× for DRAM bandwidth), so the ridge point of each new chip moves right and more kernels fall on the bandwidth slope of the roofline.
Explained in The memory wall (Architectures).
See also: Memory bandwidth, Roofline model, HBM (High Bandwidth Memory).