Glossary: System Design Beyond the Basics in Rust

Tokenize

Also written: token, tokens

Lowercasing text and splitting it into words (tokens) so they can go in an inverted index.