Attention

Also written: attend

A step where each token takes a weighted average of other tokens' value vectors, with weights set by how well its query matches their keys. It lets a token such as 'mug' pull in information from 'blue'.