Skip to content

Glossary: LLMs in Production

Max tokens

Also written: max_tokens

A cap on how many output tokens a call may produce, thinking included. Too low a cap can cut an answer off, so the service replaces answers that stopped at max_tokens.

Where the book teaches it

Related terms

Thinking tokensToken