Activation function

In neural networks, an activation function defines the output of a neuron.

The activation function takes the dot product of the input to the neuron (x) and the weights (w).

Typically activation functions are nonlinear, as that allows the network to approximate a wider variety of functions.