Skip to content

Glossary: NLP with Transformers

Dynamic quantization

Also written: int8, 8-bit

Quantization where weights are stored as 8-bit integers and each layer's inputs are also converted to 8 bits on the fly. Matrix multiplications then run in integer arithmetic.

Where the book teaches it

Related terms

Quantizationtorchao