Glossary: Serving LLMs at Scale

TPOT (time per output token)

Also written: TPOT

vLLM's name for the average time per output token, equivalent to inter-token latency.