Skip to content

Glossary: Serving LLMs at Scale

OpenAI-compatible API

Also written: OpenAI API

OpenAI's request format, which most engines copy, so the openai Python package works against llama-server or vLLM unchanged.

Where the book teaches it

Related terms

llama.cppvLLM