q8_0
Also written: --outtype q8_0
A GGUF format that stores weights in 8 bits, about half the size of 16-bit with almost no quality loss.
Also written: --outtype q8_0
A GGUF format that stores weights in 8 bits, about half the size of 16-bit with almost no quality loss.