Glossary: LLMs in Production

A/B test

Also written: A/B tests

An experiment that gives two versions of something, such as two prompts or models, to different groups of users and compares the results. Users are usually split by a hash of their ID so each person always sees the same version.