Glossary: System Design Beyond the Basics in Java

Tokenize

Also written: token, tokens

Lowercasing a text and splitting it into words, called tokens, before indexing it.