Glossary: Fine-tuning LLMs

Grouped-query attention

Attention where several query heads share keys and values, so `k_proj` and `v_proj` are smaller than `q_proj`. SmolLM2 uses it.