Training
While "running" a neural network can be done with any activation functions, we usually want to train it - i.e., adjust its parameters so that the result would be closer to what we desire.
This is commonly done by Backpropagation, which requires the activation function to be differentiable - because the adjustment of each parameter is calculated from the derivation of the activation function(s) that this parameter affects.