Glossary: Deep Learning Basics
Dead network
Also written: dead ReLU
A network whose ReLUs output 0 for every input, so no gradient flows and every prediction is the same class. Lower the learning rate and check for unscaled inputs.
Glossary: Deep Learning Basics
Also written: dead ReLU
A network whose ReLUs output 0 for every input, so no gradient flows and every prediction is the same class. Lower the learning rate and check for unscaled inputs.