Glossary

Alpha-beta (α-β) model

Beginner

A simple rule for how long sending data takes. There is a fixed delay to get started, plus a time that grows with the amount of data.

Novice

A cost model in which sending nn bytes takes α+n/B\alpha + n/B: α\alpha is a fixed per-message latency and BB is the link bandwidth (β=1/B\beta = 1/B is the time per byte). A collective’s time is the sum over its steps.

Expert

Hockney-style linear model T=α+nβT = \alpha + n\beta (+ nγ+\, n\gamma for reduction arithmetic). Ignores contention, protocol switching, pipelining granularity and congestion, but captures the latency-bound vs bandwidth-bound trade-off that drives algorithm selection.

Explained in Scale-up fabrics (Systems).

All 896 terms →