Skip to content

Glossary: Transformers Basics

Validation loss

Also written: validation set, validation

The loss measured on held-out text, here the last 10% of the file, that the model never trains on. It shows whether the model generalizes or just memorizes.

Where the book teaches it

Related terms

Cross-entropy loss