Glossary: Serving LLMs at Scale
Retry-After
An HTTP header giving the seconds a client should wait before trying again. Well-behaved clients respect it.
Glossary: Serving LLMs at Scale
An HTTP header giving the seconds a client should wait before trying again. Well-behaved clients respect it.