Skip to content

Glossary: Probability for ML

Cross-entropy (log loss)

Also written: cross-entropy, log loss, cross-entropy loss

The average of −log(probability the model gave to the true class). It punishes confident mistakes heavily and is the same as the negative log-likelihood, so training on it is maximum likelihood.

Where the book teaches it

Related terms

Negative log-likelihood (NLL)EntropyKL divergence