Glossary: Fine-tuning LLMs

Attention projections (q_proj, k_proj, v_proj, o_proj)

Also written: q_proj, k_proj, v_proj, o_proj

The four linear layers in each transformer block's attention part (query, key, value and output). LoRA can adapt just some of them, such as q_proj and v_proj, or all linear layers.