Skip to content

Glossary: Fine-tuning LLMs

LLM judge

Also written: LLM-as-judge

A strong language model given a policy, a customer's message and a reply, and asked a narrow yes-or-no question about the reply. It grades free-text outputs, such as whether Priya-style replies follow policy and answer the customer, when exact matching isn't possible.

Where the book teaches it

Related terms

Self-preference biasRule checkStructured output