Operator fusion
Beginner
Doing two or more steps in one go, so the in-between result never has to be written out to slow memory.
Novice
Combining consecutive operations of a model (say, a matrix multiply followed by an activation) so the intermediate result stays on chip instead of being written to and read back from external memory.
Expert
Raises arithmetic intensity by removing intermediate DRAM round trips. On GPUs it usually means hand-written or compiler-generated combined kernels covering a few operators; spatial dataflow chips can pipeline much longer operator chains across the fabric.
Explained in Dataflow and spatial meshes (Architectures).
See also: Arithmetic (operational) intensity, Layer pipelining (streaming dataflow).