Glossary: Serving LLMs at Scale

Kernel

Also written: FlashAttention

A small program that runs on the GPU. Engines choose fast attention kernels such as FlashAttention automatically.