Latency hiding
Beginner
Keeping busy with other work while you wait for something slow, so the waiting doesn’t cost time.
Novice
Overlapping slow operations, such as memory reads that take hundreds of clock cycles, with useful work from other threads, so the processor is rarely idle.
Expert
On GPUs, thread-level parallelism (many resident warps) plus instruction-level parallelism (independent instructions per warp). Little’s law gives the concurrency needed: latency × throughput.