Skip to content

Glossary: Transformers Basics

Bigram model

Also written: bigram

A language model that predicts the next token using only the previous token. It can be built as a table counting which character follows which in a text, such as Kettle & Crane's blurbs and messages.

Where the book teaches it

Related terms

SmoothingPerplexity