Skip to content

Glossary: Serving LLMs at Scale

Cloud Run

Google Cloud's service for running containers. It can attach an NVIDIA GPU to an instance and still scale to zero.

Where the book teaches it

Related terms

Scale to zeroCold start