Skip to content

Glossary: Transformers Basics

Padding

Also written: padding mask, attention_mask

Filler tokens added to make sequences in a batch the same length. A padding mask stops real tokens from attending to them.

Where the book teaches it

Related terms

Causal maskSpecial tokens