Glossary: Fine-tuning LLMs

LoRA (low-rank adaptation)

Also written: LoRA, low-rank adaptation

Freezing the model's weights and training two thin matrices, A and B, per layer; their product is the update. It needs far less memory than full fine-tuning and produces a small adapter.