Glossary: Deep Learning Basics
Sanity checks
Also written: overfit one batch, starting loss
Three quick checks before a long run: the starting loss should be about log(k) for k classes, the model should overfit one batch to near-zero loss, and a printed batch should have sensible shapes, dtypes and ranges.