Closed and Open divisions
Beginner
Two kinds of MLPerf results. Closed means everyone ran the same AI model, so it compares the machines. Open lets people change the model too.
Novice
In the Closed division the model and its processing must match the reference, so differences come from hardware and software. The Open division allows a different model or retraining, as long as the task and quality target are met.
Expert
Closed inference allows post-training quantization calibrated only on the supplied calibration set, with a publicly described method, but no retraining. Inference also has a Network division for systems served over a network. A public comparison of an Open result with a Closed one must say how the Open result differs.