Skip to content

Glossary: LLMs in Production

asyncio

Python's built-in library for running many waiting tasks, such as model calls, at once in one process. It lets a web service or gateway handle many requests without a thread per request.

Where the book teaches it

Related terms

async / awaitEvent loopasyncio.gather