Glossary: Fine-tuning LLMs

Reference model

The frozen model DPO compares against, usually the SFT model you started from. It keeps the tuned model from drifting far from what it already does well.