MLP (multilayer perceptron)
Also written: MLP
A small feed-forward network. In a transformer block it widens each token's vector four times, applies an activation, and narrows it back, processing each token on its own.
Also written: MLP
A small feed-forward network. In a transformer block it widens each token's vector four times, applies an activation, and narrows it back, processing each token on its own.