Glossary: System Design Beyond the Basics in JavaScript

Tokenize

Also written: token, tokens

Lowercasing text and splitting it into words, called tokens, as the first step of building a search index.