Skip to content

Glossary: Evaluating LLM Apps

Cohen's kappa

Also written: kappa

A measure of agreement between two raters that is corrected for chance: 1 is perfect and 0 is no better than chance. It shows whether a judge really beats an "always pass" judge.

Where the book teaches it

Related terms

AgreementConfusion matrixCalibration