Momentum
Keeping part of the previous step's direction during gradient descent, like a ball rolling downhill. It helps coast across plateaus.
Keeping part of the previous step's direction during gradient descent, like a ball rolling downhill. It helps coast across plateaus.