Llama
Also written: Llama 3, LLaMA
A family of open models from Meta, and an example of a modern large decoder-only transformer using RoPE, RMSNorm, SwiGLU and grouped-query attention (GQA).
Where the book teaches it
Related terms
RoPE (rotary position embedding)RMSNormSwiGLUGrouped-query attention (GQA)