Skip to content

Glossary: NLP with Transformers

Subword

Also written: subword tokens, word pieces, ##

A piece of a word that a tokenizer uses as a token. Common words stay whole, and rarer ones split into pieces, like `es ##press ##o`.

Where the book teaches it

Related terms

WordPieceBPE (byte pair encoding)