What is the output value of the network for these inputs respectively, and why? (Linear activation function is fine.)
[2, 3][-1, 2][1, 0][3, 4]
My main question is how you take the 'backwards' directed paths into account.
Artificial Intelligence Stack Exchange is a question and answer site for people interested in conceptual questions about life and challenges in a world where "cognitive" functions can be mimicked in purely digital environment. It only takes a minute to sign up.
Sign up to join this communityWhat is the output value of the network for these inputs respectively, and why? (Linear activation function is fine.)
[2, 3][-1, 2][1, 0][3, 4]
My main question is how you take the 'backwards' directed paths into account.
The neural Network in the image is a "Recurrent Neural Network"(RNN). Because of the connection leading backward from h10
to h01
, h10
has to be a "memory node" (mn), meaning it can store its value from the previous input. The basic functionality of an RNN can be seen in this animation:
In the beginning, the storage of the mn is initialized with a value, probably 0.
Now the first input is fed into network:
i0
= 2i1
= 3h00
= (i0
* 0.4) = 0.8h01
= (i1
* -0.9) + ("the stored value of h10
" * 1.2) = -2.7h10
" in the first run is 0.)h10
= (h00
* 0.85) + (h01
* -0.2) = 1.22out
= (h10
* 0.3) + (h01
* 0.1) = 0.096Now you can feed the next input through the network and use -2.7 as "the stored value of h10
" and so on. You can also add an activation function as you would for any other NN.
out
calculated later than h10
? Technically, out
and h10
is on the same layer, just like h00
is to h01
. What is the algorithm for the calculation order?
$\endgroup$
– andras
Mar 31 '18 at 13:23