Glossary: Transformers Basics

Self-attention

Attention where the queries, keys and values all come from the same sequence, so the sequence attends to itself.