Cross-attention
An extra attention layer in an encoder-decoder's decoder where queries come from the decoder but keys and values come from the encoder's output. It lets each output token ask which parts of the input it needs.
An extra attention layer in an encoder-decoder's decoder where queries come from the decoder but keys and values come from the encoder's output. It lets each output token ask which parts of the input it needs.