In [this lecture][1], the professor says that one problem with the sigmoid function is that its outputs aren't zero-centered. Are the explanation provided by the professor regarding why this is bad is that the gradient of our loss w.r.t. the weights $\frac{\partial L}{\partial w}$ which is equal to $\frac{\partial L}{\partial \sigma}\frac{\partial \sigma}{\partial w}$ will always be either negative or positive and we'll have a problem updating our weights as she show in [this slide][2], we won't be able to move in the direction of the vector $(1,-1)$. I don't understand why since she only talks about one component of our gradient and not the whole vector. I the components of the gradient of our loss will have different signs which will allow us to adjust to different directions I'm I wrong ? But the thing that I don't understand is how this property generalizes to non zero-centered functions and non-zero centered data ?


  [1]: https://youtu.be/wEoyxE0GP2M?list=PL3FW7Lu3i5JvHM8ljYj-zLfQRF3EO8sYv&t=524
  [2]: https://youtu.be/wEoyxE0GP2M?list=PL3FW7Lu3i5JvHM8ljYj-zLfQRF3EO8sYv&t=582