Skip to content

Glossary: ML in Production

Latency

Also written: p50, p95, p99

How long one request takes. Report percentiles such as p50 (median) and p95 rather than the average, which hides the slow requests.

Where the book teaches it

Related terms

Percentile