Skip to content

Glossary: Transformers Basics

Output projection

The linear layer that mixes the joined outputs of all attention heads, so what one head found can combine with another's.

Where the book teaches it

Related terms

Multi-head attention