Skip to content

Glossary: NLP with Transformers

Encoding

Also written: encoding, enc

What a tokenizer returns: a dictionary of `input_ids`, `attention_mask` and sometimes `token_type_ids`, with one entry per token.

Where the book teaches it

Related terms

Attention maskToken type IDsSpecial tokens