Skip to content

Glossary: LLMs in Production

Output speed

Also written: tokens per second

How many tokens per second a model writes once it has started. It only means something for long answers.

Where the book teaches it

Related terms

Time to first token (TTFT)Latency