Glossary: System Design Beyond the Basics in C++

Tokenize

Also written: token, tokens

Lowercasing text and splitting it into words, called tokens, before building an inverted index.