1
$\begingroup$

Raul Rojas' Neural Networks A Systematic Introduction, section 8.2.1 calculates the variance of the output of a hidden neuron.

Raul Rojas says that "for binary vectors we have $E[x_i^2] = \frac{1}{3}$" where $x_i$ is the input value transported through each edge to a node.

I don't quite get how he reaches this result.

Thank you for your time :)

$\endgroup$
1
  • $\begingroup$ The integral from 0 to 1 of x^2dx is 1/3, but isn't this related to a continuous random variable, instead of a binary one? $\endgroup$ Apr 24, 2019 at 4:47

1 Answer 1

2
$\begingroup$

Some lines above the author says

By the law of large numbers we can also assume that the total input to the node has a Gaussian distribution

hence we can assume $X \sim \mathcal{N}(0,1)$ with the $X$ domain being continuous

Then he says the input vector is assumed to be binary which changes the domain from continuous to discrete so we can discretize it assuming $-1 \le X \le 1$ is mapped into zero and $X< -1 $ and $X > 1$ are mapped to 1

Finally according to the 68-95-99.7 Rule we can compute

$$ E(X^2) = P(X=1) \cdot 1^2 + P(X=0) \cdot 0^2 = 0.32 $$

Finally probably the author rounds this up to $\frac{1}{3} \simeq 0.33$

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .