All-to-all
Beginner
Every chip sends a different message to every other chip at the same time.
Novice
A collective in which each participant sends a distinct block to each of the others: a full personalized exchange. Mixture-of-experts models use it to route tokens to the chips that hold each expert.
Expert
Each participant sends and receives of its buffer, but every pair exchanges distinct data, so it stresses bisection bandwidth and incast; it cannot be shortened by pipelining around a ring the way reductions can.
Explained in Scale-up fabrics (Systems).