Attention weights
Also written: weights (attention)
The softmax of the scaled query-key scores: for each token, positive numbers that add up to 1 and say how much it takes from each other token's value.
Also written: weights (attention)
The softmax of the scaled query-key scores: for each token, positive numbers that add up to 1 and say how much it takes from each other token's value.