Glossary: NLP with Transformers
Quantization
Also written: quantize_, quantized
Storing weights as 8-bit integers instead of 32-bit floats. It made DistilBERT 15% to 40% faster here without changing an answer.
Glossary: NLP with Transformers
Also written: quantize_, quantized
Storing weights as 8-bit integers instead of 32-bit floats. It made DistilBERT 15% to 40% faster here without changing an answer.