Glossary: Serving LLMs at Scale
A Kubernetes tool that scales deployments on a metric such as a Prometheus value.
KubernetesPrometheus