Glossary: Serving LLMs at Scale

NVLink

A fast link between GPUs in one machine, needed for tensor parallelism because GPUs exchange data at every layer.