bfloat16
Also written: bf16
A 16-bit number format with float32's range but less precision, using 2 bytes per parameter. It is the usual format for storing a frozen base model during LoRA fine-tuning.
Also written: bf16
A 16-bit number format with float32's range but less precision, using 2 bytes per parameter. It is the usual format for storing a frozen base model during LoRA fine-tuning.