Glossary: Serving LLMs at Scale

Disaggregated serving

Running prefill and decode on separate GPUs. It is meant for large fleets.