Skip to content

Glossary: Transformers Basics

RNN (recurrent network)

Also written: RNN, recurrent neural network, recurrent network

A network that reads one token at a time while carrying a hidden state. It squeezes the whole past through one vector and can't compute positions in parallel.

Where the book teaches it

Related terms

Hidden stateAttention