Learning rate
Also written: learning_rate, lr
The size of each update during training. About 2e-4 is typical for LoRA, higher than for full fine-tuning; too high makes the loss jump, too low makes it barely move.
Also written: learning_rate, lr
The size of each update during training. About 2e-4 is typical for LoRA, higher than for full fine-tuning; too high makes the loss jump, too low makes it barely move.