All-gather
Beginner
Each chip has one piece of a puzzle; at the end, every chip has the whole puzzle.
Novice
A collective in which each participant contributes a block and every participant ends up with all the blocks, concatenated.
Expert
Moves of the final buffer per participant at minimum; no arithmetic. The second half of a bandwidth-optimal all-reduce.