Glossary: Transformers Basics

SentencePiece

Also written: unigram

A tokenizer library and method used by T5, with a 32,000-token vocabulary.