Glossary: Serving LLMs at Scale
Time to first token (TTFT)
Also written: TTFT
How long a user waits until the first word of the answer appears. It is mostly prefill plus any queueing.
Glossary: Serving LLMs at Scale
Also written: TTFT
How long a user waits until the first word of the answer appears. It is mostly prefill plus any queueing.