Glossary: Serving LLMs at Scale

FastAPI

Also written: uvicorn

A Python web framework used to build router.py; uvicorn is the server that runs it.