Prompt caching
Also written: cache_control, cache_read_input_tokens
The provider keeps a request's prefix for a few minutes and charges a tenth of the price to read it again. It made Dana's three-call run 39% cheaper.
Also written: cache_control, cache_read_input_tokens
The provider keeps a request's prefix for a few minutes and charges a tenth of the price to read it again. It made Dana's three-call run 39% cheaper.