Glossary: Transformers Basics
The vector a recurrent network carries from step to step, summarizing everything it has read so far.
RNN (recurrent network)