Quantization

Also written: quantized, quant

Storing each weight with fewer bits, rounded to a small set of values, so the file is smaller and faster to read. Q4_K_M keeps about 4.9 bits per weight with a small quality loss and is the usual default.