Glossary: AI System Design
How long a user waits, measured as time to first token and total time, set with percentiles rather than averages.
TTFT (time to first token)p95Latency budget