Skip to content

Glossary: LLMs in Production

Queue / workers

Also written: queue, workers

A queue holds jobs nobody waits for, such as tagging yesterday's chats, and workers take them off at a steady pace. This smooths bursts and suits batch APIs.

Where the book teaches it

Related terms

Batch APIAmazon SQS