What is the output value of the network for these inputs respectively, and why? (Linear activation function is fine.)
[2, 3][-1, 2][1, 0][3, 4]
My main question is how you take the 'backwards' directed paths into account.
$\begingroup$
$\endgroup$
$\begingroup$
$\endgroup$
The neural Network in the image is a "Recurrent Neural Network"(RNN). Because of the connection leading backward from h10 to h01, h10 has to be a "memory node" (mn), meaning it can store its value from the previous input. The basic functionality of an RNN can be seen in this animation:
Your example:
In the beginning, the storage of the mn is initialized with a value, probably 0.
Now the first input is fed into network:
i0= 2i1= 3h00= (i0* 0.4) = 0.8h01= (i1* -0.9) + ("the stored value ofh10" * 1.2) = -2.7
("the stored value ofh10" in the first run is 0.)h10= (h00* 0.85) + (h01* -0.2) = 1.22out= (h10* 0.3) + (h01* 0.1) = 0.096
Now you can feed the next input through the network and use -2.7 as "the stored value of h10" and so on. You can also add an activation function as you would for any other NN.
-
$\begingroup$ Why is
outcalculated later thanh10? Technically,outandh10is on the same layer, just likeh00is toh01. What is the algorithm for the calculation order? $\endgroup$ – andras Mar 31 '18 at 13:23
