Glossary: Serving LLMs at Scale
SGLang
Also written: sglang serve
An open-source GPU serving engine from the LMSYS group, known for RadixAttention and fast structured output.
Glossary: Serving LLMs at Scale
Also written: sglang serve
An open-source GPU serving engine from the LMSYS group, known for RadixAttention and fast structured output.