Skip to content

Glossary: Transformers Basics

Query, key, value

Also written: query, key, value, Q, K, V, queries, keys, values

The three vectors attention gives each token: a query for what it is looking for, a key for what it offers for matching, and a value for the information it passes on if chosen.

Where the book teaches it

Related terms

AttentionScaled dot-product attention