DPOTrainer
Also written: DPOConfig
TRL's trainer for DPO, which expects data with a prompt plus chosen and rejected completions. With a PEFT adapter and no separate reference, it uses a frozen copy of the adapter as the reference.
Also written: DPOConfig
TRL's trainer for DPO, which expects data with a prompt plus chosen and rejected completions. With a PEFT adapter and no separate reference, it uses a frozen copy of the adapter as the reference.