Glossary: Serving LLMs at Scale

AWQ (activation-aware weight quantization)

Also written: AWQ

A 4-bit weight quantization method that protects the few weights that matter most, judged by the activations flowing through them. It needs a calibration set.