Skip to content

Glossary: LLMs in Production

asyncio.timeout

Also written: reschedule()

A context manager that raises an error if the block takes too long. The gateway uses it for the first-token deadline, then moves the deadline out with reschedule() when text arrives.

Where the book teaches it

Related terms

TimeoutTime to first token (TTFT)