Rank (r)
Also written: rank, r
The inner size of LoRA's two thin matrices, meaning how many directions the update can use. 8 to 64 is typical, and 16 is a good start for a narrow task.
Also written: rank, r
The inner size of LoRA's two thin matrices, meaning how many directions the update can use. 8 to 64 is typical, and 16 is a good start for a narrow task.