Skip to content

Glossary: AI System Design

Rate limit

Also written: RPM, ITPM, OTPM, 429, rate limits

A provider's cap per account on requests (RPM), input tokens (ITPM) or output tokens (OTPM) per minute. Exceeding one returns a 429 error, and the limit is shared by every product on the account.

Where the book teaches it

Related terms

Fair queueModel gateway