Skip to content

Glossary: Serving LLMs at Scale

asyncio

Also written: async def, await, AsyncOpenAI

Python's built-in way to wait on many network requests at once. An async function pauses at each await while others run, which is how loadtest.py sends many requests.

Where the book teaches it

Related terms

Load test