Glossary: NLP with Transformers
Word IDs
Also written: word_ids, word_ids()
A list giving each token's word number in the original text, or `None` for special tokens. It lets you align word-level labels with subword tokens.
Glossary: NLP with Transformers
Also written: word_ids, word_ids()
A list giving each token's word number in the original text, or `None` for special tokens. It lets you align word-level labels with subword tokens.