Glossary: Serving LLMs at Scale
Container
Also written: container image, Dockerfile, docker run
A packaged, isolated copy of a program with its dependencies. Here the engine goes in an image and the weights come in through a volume.
Glossary: Serving LLMs at Scale
Also written: container image, Dockerfile, docker run
A packaged, isolated copy of a program with its dependencies. Here the engine goes in an image and the weights come in through a volume.