In neural networks, an activation function defines the output of a neuron.
The activation function takes the dot product of the input to the neuron () and the weights ().
Typically activation functions are nonlinear, as that allows the network to approximate a wider variety of functions.