Glossary: Deep Learning Basics

Learning rate

Also written: lr

The size of each optimizer step. Too small barely moves; too large overshoots and the loss explodes to `nan`. Try values a factor of 10 apart.