Skip to content

Glossary: Probability for ML

Surprise

−log p: how unexpected an outcome of probability p is. Likely outcomes surprise little and rare ones a lot; entropy is the average surprise.

Where the book teaches it

Related terms

EntropyBits, nats