Bulk synchronous parallel (BSP)
Beginner
A way of organizing teamwork in rounds: everyone works alone, then everyone swaps results, then everyone waits until all are done before the next round.
Novice
A parallel programming model that runs in supersteps: each processor computes on its local data, then all processors exchange data, then a barrier makes everyone wait until the exchange is finished.
Expert
Separates compute from communication in time, so the exchange can be scheduled at compile time with no contention surprises; the cost is that compute and communication do not overlap within a superstep and every barrier waits for the slowest participant.
Explained in Dataflow and spatial meshes (Architectures).
See also: Processing element (PE).