Glossary: Building with LLMs Basics
Prefix cache
A cache that matches the start of a prompt, so what never changes (tools, then the system prompt) goes first and what changes goes last. The prefix must match exactly.
Glossary: Building with LLMs Basics
A cache that matches the start of a prompt, so what never changes (tools, then the system prompt) goes first and what changes goes last. The prefix must match exactly.