Glossary: Serving LLMs at Scale
SGLang's way of keeping cached prefixes in a tree for reuse.
Prefix cachingSGLang