Glossary: Deep Learning Basics
Gradient scaler
Also written: GradScaler, torch.amp.GradScaler
Multiplies the loss before `backward` in float16 training so tiny gradients don't round to zero, then unscales them before the optimizer step.
Glossary: Deep Learning Basics
Also written: GradScaler, torch.amp.GradScaler
Multiplies the loss before `backward` in float16 training so tiny gradients don't round to zero, then unscales them before the optimizer step.