Glossary: NLP with Transformers
Encoding
Also written: encoding, enc
What a tokenizer returns: a dictionary of `input_ids`, `attention_mask` and sometimes `token_type_ids`, with one entry per token.
Glossary: NLP with Transformers
Also written: encoding, enc
What a tokenizer returns: a dictionary of `input_ids`, `attention_mask` and sometimes `token_type_ids`, with one entry per token.