Binary vector expected value

Raul Rojas' Neural Networks A Systematic Introduction, section 8.2.1 calculates the variance of the output of a hidden neuron.

Raul Rojas says that "for binary vectors we have $$E[x_i^2] = \frac{1}{3}$$" where $$x_i$$ is the input value transported through each edge to a node.

I don't quite get how he reaches this result.

Thank you for your time :)

• The integral from 0 to 1 of x^2dx is 1/3, but isn't this related to a continuous random variable, instead of a binary one? Commented Apr 24, 2019 at 4:47

Some lines above the author says

By the law of large numbers we can also assume that the total input to the node has a Gaussian distribution

hence we can assume $$X \sim \mathcal{N}(0,1)$$ with the $$X$$ domain being continuous

Then he says the input vector is assumed to be binary which changes the domain from continuous to discrete so we can discretize it assuming $$-1 \le X \le 1$$ is mapped into zero and $$X< -1$$ and $$X > 1$$ are mapped to 1

Finally according to the 68-95-99.7 Rule we can compute

$$E(X^2) = P(X=1) \cdot 1^2 + P(X=0) \cdot 0^2 = 0.32$$

Finally probably the author rounds this up to $$\frac{1}{3} \simeq 0.33$$