Beta
Also written: beta
In DPO, the setting that controls how strongly the model is held close to the reference model. Higher beta means smaller moves; TRL's default is 0.1.
Where the book teaches it
Related terms
DPO (direct preference optimization)Reference modelRewards (in DPO logs)