Glossary: Serving LLMs at Scale
OpenAI-compatible API
Also written: OpenAI API
OpenAI's request format, which most engines copy, so the openai Python package works against llama-server or vLLM unchanged.
Glossary: Serving LLMs at Scale
Also written: OpenAI API
OpenAI's request format, which most engines copy, so the openai Python package works against llama-server or vLLM unchanged.