Skip to content

Glossary: NLP with Transformers

Special tokens

Also written: [CLS], [SEP], [PAD], [MASK], [UNK]

Reserved tokens with jobs: `[CLS]` starts an input, `[SEP]` ends or separates texts, `[PAD]` fills short texts, `[MASK]` hides a word and `[UNK]` stands for unknown text.

Where the book teaches it

Related terms

Padding, truncationToken type IDs