DPO (direct preference optimization)
Also written: DPO, direct preference optimization
A way to tune a model on human comparisons of answers directly, without a separate reward model or reinforcement learning. Llama 3's post-training uses SFT followed by DPO.
Where the book teaches it
Related terms
RLHF (reinforcement learning from human feedback)Instruction tuning (SFT)Preference tuning