GPU (graphics processing unit)
Beginner
A chip with thousands of small math units that all work at once. It was built to draw video-game graphics and turned out to be great at AI math too.
Novice
A processor made of many simple cores that run the same program on huge amounts of data in parallel. Originally fixed-function graphics hardware, it became programmable in the 2000s and now runs most AI training.
Expert
A throughput processor: dozens to a few hundred multithreaded cores (SMs or CUs), each keeping thousands of threads resident to hide memory latency, plus matrix units for dense linear algebra and a high-bandwidth memory system.
Explained in SIMT and GPUs (Architectures).
See also: Streaming multiprocessor (SM) / compute unit (CU), SIMT (single instruction, multiple threads).