Skip to content

Glossary: Serving LLMs at Scale

TGI (Text Generation Inference)

Also written: TGI

Hugging Face's engine for serving LLMs, now in maintenance mode, with its documentation recommending vLLM, SGLang, llama.cpp or MLX instead. It is not a good choice for starting new projects.

Where the book teaches it

Related terms

vLLMSGLang