Skip to content

Glossary: LLMs in Production

Retry budget

Also written: RetryBudget

A cap on retries as a share of recent calls, such as 10 plus one per 10 calls in the last 60 seconds. In a big outage most calls then fail fast.

Where the book teaches it

Related terms

Retry stormSliding window