Glossary: Deep Learning Basics
SGD
Also written: stochastic gradient descent, SGD optimizer
Stochastic gradient descent: each step moves the parameters by `lr * grad` computed from one random batch. "Stochastic" refers to using random batches.
Glossary: Deep Learning Basics
Also written: stochastic gradient descent, SGD optimizer
Stochastic gradient descent: each step moves the parameters by `lr * grad` computed from one random batch. "Stochastic" refers to using random batches.