Surprise
−log p: how unexpected an outcome of probability p is. Likely outcomes surprise little and rare ones a lot; entropy is the average surprise.
−log p: how unexpected an outcome of probability p is. Likely outcomes surprise little and rare ones a lot; entropy is the average surprise.