Skip to content

Glossary: AI System Design

Judged sample

Also written: LLM judge

A share of live calls (2% for support) graded each day by a stronger model, with an alert if the score drops.

Where the book teaches it

Related terms

Shadow testCanaryBatch API