Skip to content

Glossary: NLP with Transformers

Attention mask

Also written: attention_mask

A list of 1s and 0s given to the model with a batch. A 1 marks a real token and a 0 marks padding, so attention ignores the `[PAD]` positions.

Where the book teaches it

Related terms

Padding, truncationSpecial tokens