Glossary: LLMs in Production

asyncio.Semaphore

Also written: Semaphore

Holds a fixed number of slots: acquire() waits for a free one and release() gives it back. It caps how many calls are in progress at once.