Glossary: Calculus for ML

Mini-batch / SGD

Also written: mini-batch, mini-batches, stochastic gradient descent, SGD

Taking each gradient step using a small random batch of the data, such as 5 of Sam's 10 weeks. SGD stands for stochastic gradient descent, where 'stochastic' means random. It is cheaper per step and its randomness helps shake training off saddle points.