Scaling laws
Also written: scaling law
The finding that a language model's loss falls predictably as a power law in parameters, data and compute. Labs use it to predict a big model's quality from small experiments.
Also written: scaling law
The finding that a language model's loss falls predictably as a power law in parameters, data and compute. Labs use it to predict a big model's quality from small experiments.