Memory hierarchy
Beginner
Layers of memory from tiny and very fast, right next to the processor, to huge and slow, far away.
Novice
The layers of memory in a computer, from fastest and smallest to slowest and largest: registers inside the processor, then on-chip caches or SRAM (fast memory built from transistors on the chip itself), then DRAM main memory on separate chips. Each level is bigger, slower and cheaper per bit than the one above. It works because programs tend to reuse the same and nearby data.
Expert
Each level is sized by capacity, access time, bandwidth and energy per access, plus rules for which copies are kept (inclusion, replacement) and how copies stay consistent (coherence). Each on-chip level becomes blocks of SRAM in the floorplan, so these choices set much of the die area.
Explained in Architecture (Design Flow).
See also: Cache, DRAM, AMAT (average memory access time).