Glossary: Serving LLMs at Scale

llm-compressor

Also written: AutoAWQ

The vLLM project's tool for quantizing your own models. The older AutoAWQ library is deprecated.