Skip to content

Glossary: Serving LLMs at Scale

Time to first token (TTFT)

Also written: TTFT

How long a user waits until the first word of the answer appears. It is mostly prefill plus any queueing.

Where the book teaches it

Related terms

PrefillInter-token latency (ITL)Latency