Weights (parameters)
Beginner
The numbers a neural network learned during training. They are its knowledge, and there can be billions of them.
Novice
The learned values inside a neural network’s layers. A model’s size is usually given as its parameter count, such as 7 billion; at 2 bytes each that is 14 GB to store.
Expert
Fixed during inference and updated every step during training. Whether they can be reused across many inputs per fetch (batching) decides much of a layer’s arithmetic intensity.