Skip to content

Glossary: Transformers Basics

Language-model head

Also written: LM head

The last linear layer of a language model, turning each vector into one score per vocabulary token.

Where the book teaches it

Related terms

LogitsWeight tying