Skip to content

Glossary: LLMs in Production

Rate limit

Also written: rate limits

A provider's cap on requests or tokens per minute. Exceeding it brings 429 errors, so a service should also limit its own call rate to stay under it.

Where the book teaches it

Related terms

Token bucketRate limiter