Glossary: Serving LLMs at Scale

HTTP status codes

Also written: 200, 400, 401, 429, 503, 429 Too Many Requests, 503 Service Unavailable, 401 Unauthorized

Numbers servers return with a reply: 200 OK; 400 request too big (exceeds context); 401 missing or invalid API key; 429 Too Many Requests ('you, slow down', a user over their limit); 503 Service Unavailable ('we're full or loading, try again soon').