Activation function

Also written: activation

A function such as tanh or ReLU that is applied to each hidden unit's score. It bends the output, which lets a network fit curves a straight line can't.