Glossary: Fine-tuning LLMs

DPO (direct preference optimization)

Also written: DPO, direct preference optimization

Preference tuning done in one ordinary training loop. It pushes the model to make the chosen answer likelier than the rejected one, relative to a frozen reference model.