Glossary: Deep Learning Basics
Gradient clipping
Also written: clip_grad_norm_, nn.utils.clip_grad_norm_
Scaling gradients down whenever their combined size (the norm) is too large, to stop exploding gradients in deeper networks. `nn.utils.clip_grad_norm_` does it, is called between `backward` and `step`, and returns the norm before clipping.