Skip to content

Glossary: Fine-tuning LLMs

Alpha (lora_alpha)

Also written: alpha, lora_alpha

LoRA's scale setting. The adapter's update is multiplied by alpha / r, so it controls how strongly the adapter speaks relative to the frozen weights.

Where the book teaches it

Related terms

Rank (r)LoRA (low-rank adaptation)