Cache hit rate
Also written: hit rate
The share of prompt tokens served from the provider's cache: cache reads divided by cache reads plus cache writes plus uncached input. It should be above 90% with steady traffic.
Also written: hit rate
The share of prompt tokens served from the provider's cache: cache reads divided by cache reads plus cache writes plus uncached input. It should be above 90% with steady traffic.