Two common activation functions used in deep learning are the hyperbolic tangent function and the sigmoid activation function. I understand that the hyperbolic tangent is just a rescaling and translation of the sigmoid function:
$\tanh(z) = 2\sigma(z) - 1$.
Is there a significant difference between these two activation functions, and in particular, when is one preferable to the other?
I realize that in some cases (like when estimating probabilities) outputs in the range of $[0,1]$ are more convenient than outputs that range from $[-1,1]$. I want to know if there are differences other than convenience which distinguish the two activation functions.