Layer normalization
Also written: LayerNorm, layer norm, nn.LayerNorm
Rescaling each token's vector to mean 0 and standard deviation 1, then applying a learned scale and shift. It keeps values steady as they pass through many layers.
Also written: LayerNorm, layer norm, nn.LayerNorm
Rescaling each token's vector to mean 0 and standard deviation 1, then applying a learned scale and shift. It keeps values steady as they pass through many layers.