Skip to content

Glossary: Deep Learning Basics

Sanity checks

Also written: overfit one batch, starting loss

Three quick checks before a long run: the starting loss should be about log(k) for k classes, the model should overfit one batch to near-zero loss, and a printed batch should have sensible shapes, dtypes and ranges.

Where the book teaches it

Related terms

Cross-entropyOverfitting