Glossary: System Design Beyond the Basics

Tokenize

Also written: token, tokens

Lowercasing text and splitting it into words, called tokens, as the first step of building an inverted index.