Glossary: Serving LLMs at Scale

Serving

Answering many people over a network at the same time, all day, optimizing both everyone's wait and the cost per answer.