Skip to content

Glossary: LLMs in Production

Batch API

Also written: batch, batch processing, batches

A provider API that takes many requests and processes them within 24 hours at half price. Kettle & Crane uses it for the nightly chat tagging that nobody waits for.

Where the book teaches it

Related terms

Flex processingQueue / workers