Skip to content

Glossary: Evaluating LLM Apps

Agreement

Also written: observed agreement

The share of answers where a judge and a human gave the same label. It looks high when most answers pass, so it is usually paired with kappa.

Where the book teaches it

Related terms

Cohen's kappaConfusion matrixCalibration