In a lot of explanations online for Xavier Initialization, I see the following:
With each passing layer, we want the variance to remain the same. This helps us keep the signal from exploding to a high value or vanishing to zero. In other words, we need to initialize the weights in such a way that the variance remains the same for x and y. This initialization process is known as Xavier initialization.
However, the intuition behind why var(output) should equal var(inputs) is never explained. Does anyone know why intuitively var(output) should equal var(inputs)?