Skip to content

Glossary: Building with LLMs Basics

Token

Also written: tokens

A chunk of text, often a word or part of a word. For English, about 4 characters or three quarters of a word; you pay per token and limits are counted in tokens.

Where the book teaches it

Related terms

TokenizerContext windowMTok