Glossary: Serving LLMs at Scale
Cloud Run
Google Cloud's service for running containers. It can attach an NVIDIA GPU to an instance and still scale to zero.
Glossary: Serving LLMs at Scale
Google Cloud's service for running containers. It can attach an NVIDIA GPU to an instance and still scale to zero.