Skip to content

Glossary: Evaluating LLM Apps

A/B test

Also written: A/B testing

An experiment that sends some users to version A and others to version B at the same time, then compares outcomes such as thumbs-up rate. Kettle & Crane split users by ID so nobody sees both versions.

Where the book teaches it

Related terms

Two-proportion z-testSample ratio mismatchp-value