Skip to content

Glossary: Fine-tuning LLMs

Entropy (in the logs)

Also written: entropy

A training-log value showing how spread out the model's predicted probabilities are. Low entropy means the model is confident.

Where the book teaches it

Related terms

Mean token accuracyTraining loss