0
$\begingroup$

In most of the online materials that I've read, the equations of RNNs are shown only for a single layer RNN with the output going through softmax (for a many-to-one architecture).

I am trying to find out how RNNs behave in the case of having multiple layers. Let's take two layers as an example (as shown below).

  • U1 = Weights on the input going into layer 1 - shape (3,4)
  • V1 = Weights on the hidden state going into layer 1 - shape (4,4)
  • U2 = Weights on the input going into layer 2 - shape (4,3)
  • V2 = Weights on the hidden state going in layer 2 - shape (3,3)
  • W = Weights on the final output - shape (3, 5)
  • X1, X2, X3 are inputs to layer 1 - shape (3,1)

Since the final output Y is in the form of probabilities, I understand that it has to go through a softmax function.

My Question is regarding the outputs of layer 1 fed into layer 2. Does the RNN apply softmax to them as well?

enter image description here

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.