Glossary: Serving LLMs at Scale
Kubernetes
Also written: kubectl, YAML
A system that runs containers across a cluster and keeps them running, driven by YAML descriptions. A GPU is requested like memory, as nvidia.com/gpu.
Glossary: Serving LLMs at Scale
Also written: kubectl, YAML
A system that runs containers across a cluster and keeps them running, driven by YAML descriptions. A GPU is requested like memory, as nvidia.com/gpu.