Skip to content

Glossary: Building with LLMs Basics

Batch API

Also written: Message Batches API, batches, batch

A service that takes many requests at once, processes them when it has capacity and charges 50% of the normal price. It suits jobs like re-sorting last month's emails that don't need answers in seconds.

Where the book teaches it

Related terms

custom_idPrompt caching