Skip to content

Glossary: Evaluating LLM Apps

Inconsistent verdict

Also written: inconsistent

A pairwise result that changes when the two answers swap places, such as a win in one order and a tie or loss in the other. It is a sign of position bias and is counted separately rather than as a win.

Where the book teaches it

Related terms

Pairwise judgingPosition bias