Elementwise operation
Beginner
A simple step done to each number on its own, such as turning negatives into zero.
Novice
An operation that produces each output value from the matching input value(s) alone, such as an activation function or adding two tensors. It does about one operation per value read.
Expert
Intensity well under 1 FLOP/byte, so always memory-bound unless fused into a neighboring kernel. Reductions (softmax, layer norm) are only slightly better.
Explained in What the workload needs (Architectures).
See also: Kernel fusion, Memory-bound.