CUDA
Beginner
NVIDIA’s programming system for writing general-purpose programs that run on its GPUs.
Novice
A programming model and toolkit, introduced in 2007, that extends C++ so programmers write a function for one thread and launch it across thousands of GPU threads. AMD’s HIP is a closely matching, portable alternative.
Expert
Grid → block → warp → thread hierarchy, explicit shared memory and barriers, plus libraries (cuBLAS, cuDNN, CUTLASS). Its install base and libraries are a large part of why GPUs dominate AI software.
Explained in SIMT and GPUs (Architectures).
See also: Kernel (GPU), Thread block (workgroup).