Glossary: Serving LLMs at Scale
Concurrency sweep
A load test that repeats a closed-loop run at rising concurrency (1, 2, 4, 8, 16 users) to see where throughput flattens and latency climbs.
Glossary: Serving LLMs at Scale
A load test that repeats a closed-loop run at rising concurrency (1, 2, 4, 8, 16 users) to see where throughput flattens and latency climbs.