Full fine-tuning
Updating every weight of the model. It is the most flexible option but needs about 16 bytes of memory per parameter, plus activations.
Where the book teaches it
Related terms
LoRA (low-rank adaptation)Optimizer statesMixed precision (mixed-precision)