Kernel (GPU)
Beginner
A small program sent to the GPU that runs many thousands of times at once, once for each piece of data.
Novice
A function launched on the GPU across a grid of threads. Each thread runs the same code and uses its thread number to pick which data to work on.
Expert
A device function launched over a grid of thread blocks. Its per-thread register count, per-block shared memory and block size set occupancy; one layer of a neural network is often one or a few kernels.