Prompt caching
Also written: prompt cache, cache, caching
The provider keeps its processed copy of a prompt's start and reuses it for requests beginning with exactly the same text. For Kettle & Crane it cuts the help center's cost to a tenth.
Also written: prompt cache, cache, caching
The provider keeps its processed copy of a prompt's start and reuses it for requests beginning with exactly the same text. For Kettle & Crane it cuts the help center's cost to a tenth.