Gradient check
Also written: grad_check
Comparing your hand-derived gradients with numerical ones to catch backpropagation bugs. Such bugs otherwise fail silently, because the model still trains, just badly.
Also written: grad_check
Comparing your hand-derived gradients with numerical ones to catch backpropagation bugs. Such bugs otherwise fail silently, because the model still trains, just badly.