Glossary: Deep Learning Basics

Learning-rate schedule

Also written: scheduler, schedulers, StepLR, CosineAnnealingLR, OneCycleLR, lr_scheduler

A plan that starts with a big learning rate and shrinks it during training. `StepLR` multiplies it by `gamma` every few epochs; `CosineAnnealingLR` glides it toward zero along a half cosine.