Skip to content

Glossary: Serving LLMs at Scale

Hash

Also written: fingerprint

A fingerprint computed from data. vLLM hashes each full block of tokens with everything before it to find reusable cached blocks; the router hashes a user name to pick a home replica.

Where the book teaches it

Related terms

Prefix cachingAffinity