Offline evaluation (test set)
Also written: test set, offline eval, release gate
A fixed set of questions with known good answers, graded by code and an LLM judge before every release; a score drop beyond the margin blocks the release.
Also written: test set, offline eval, release gate
A fixed set of questions with known good answers, graded by code and an LLM judge before every release; a score drop beyond the margin blocks the release.