Skip to content

Glossary: Serving LLMs at Scale

Knee

The point in a concurrency sweep where latency shoots up because requests start queueing. Run below it.

Where the book teaches it

Related terms

Concurrency sweep