Prefix caching

Also written: prefix

Keeping the KV cache of a prompt's beginning and reusing it for any new request that starts with the same tokens, skipping their prefill. Put what never changes first and what changes last.