Skip to content

Glossary: Transformers Basics

GPT

Also written: GPT-2, GPT-3, generative pretrained transformer

A family of decoder-only transformer language models from OpenAI that generate text by predicting the next token. GPT-2's architecture, scaled down to a tiny size, is a common model for learning how these systems work.

Where the book teaches it

Related terms

Decoder-only modelWeight tying