Skip to content

Glossary: Evaluating LLM Apps

Flaky case

Also written: flaky

A case that passes on some tries and fails on others because model replies vary. Kettle & Crane's c36 was the only flaky one.

Where the book teaches it

Related terms

pass^kMust-pass case