Skip to content

Glossary: LLMs in Production

Cost per 1,000 questions

Also written: per 1,000, cost per 1,000 answers

A request's cost scaled up to 1,000 questions, used to compare designs because a single request costs only a fraction of a cent. It is easy to multiply by traffic and to set alerts on.

Where the book teaches it

Related terms

MultipliersPrompt caching