Skip to content

Glossary: Serving LLMs at Scale

Container

Also written: container image, Dockerfile, docker run

A packaged, isolated copy of a program with its dependencies. Here the engine goes in an image and the weights come in through a volume.

Where the book teaches it

Related terms

VolumeKubernetes