Skip to content

Glossary: LLMs in Production

Semantic cache

A response cache that also reuses answers for similarly worded questions. It can confidently serve a wrong answer when a number or detail changes, such as a $60 order versus a $40 one.

Where the book teaches it

Related terms

Response cacheCosine similarity