Skip to content

Glossary: LLMs in Production

Deadline

The total time retries may take. When the next wait would pass it, the gateway stops retrying and moves on to a fallback.

Where the book teaches it

Related terms

Retry budgetFallback