Glossary: Running Open Models Locally

Grouped-query attention

Also written: GQA

A memory-saving design where a model has fewer KV heads than attention heads. It makes the KV cache smaller.