Glossary: Running Open Models Locally
Time to first token (TTFT)
Also written: TTFT
The seconds until the first token of the reply appears: loading (if needed) plus prefill. It is what a user waiting on a streamed reply notices first.
Glossary: Running Open Models Locally
Also written: TTFT
The seconds until the first token of the reply appears: loading (if needed) plus prefill. It is what a user waiting on a streamed reply notices first.