Skip to content

Glossary: Transformers Basics

BERT

An encoder-only transformer from 2018 trained by masked language modeling. It suits classification, tagging and search more than text generation.

Where the book teaches it

Related terms

Encoder-only modelMasked language modelingSpecial tokens