1
$\begingroup$

RNN and LSTM models have many architectures that can be modified. We can also compose their input and output data. However, in the examples that I found on the web, the inputs and outputs of RNNs/LSTMs are usually sequences.

Let's say we have a 3-column dataset:

data= np.array([[1.022 0.94  1.278]
                [2.096 1.404 2.035]
                [1.622 2.348 1.909]
                [1.678 1.638 1.742]
                [2.279 1.878 2.045]])

where the first two columns contain the inputs (features) and the third one contains the labels.

Usually, when modeling with feedforward neural networks (FFNNs), the input and output look like this:

Input:

x_input = np.vstack((data[:, 0], data[:, 1])).reshape(5, 2)

[[1.022 2.096]
 [1.622 1.678]
 [2.279 0.94 ]
 [1.404 2.348]
 [1.638 1.878]]

Output:

y_output = np.vstack((data[:, 2])).reshape(5, 1)

[[1.278]
 [2.035]
 [1.909]
 [1.742]
 [2.045]]

When modeling with RNN, the input and output are:

Input:

[[1.022 0.94  1.278]
 [2.096 1.404 2.035]
 [1.622 2.348 1.909]

Output (as a sequence):

 [1.678 1.638 1.742]
 [2.279 1.878 2.045]]

I would like to ask: Is it possible to model the input and output as an ANN model when modeling with RNN? Would it be correct?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

Yes, it is possible. What you have shown in case of ANN is what happens in a regression model using NNs. What you have shown in case of RNN is what happens when you are doing sequence-to-sequence translation (like French to English).

If you want to get single values like in case of ANN, suppose you are doing regression, then, in the end, you will flatten the features aggregated by RNN (in case of Tensorflow, use Flatten layer and in case of PyTorch, you can directly do it). It should be then followed by a dense layer of 3 (in case of Tensorflow) or linear layer of 3 (in case of PyTorch), if I am talking about your example.

Since, you have shown values above 1, I presume you are doing some kind of regression. But, it would be good idea to normalize your outputs in case of regression, it makes the optimization easier.

If you want to classification then in the last layer, use Dense layer with softmax (in case of Tensorflow) and softmax followed by a linear layer (in case of PyTorch).

Improve this answer
$\endgroup$
6
  • $\begingroup$ f(x, y, z) = ans, if you are trying to approximate the f here. Then, you don't need a RNN. A simple Dense layer network would work here. Or is there a concept of timestep? In that you will have to decide upon a window size. For flatten layer: tensorflow.org/api_docs/python/tf/keras/layers/Flatten. Look at the example in here. Basically, RNN would give 3D output while Dense layer requires 2D input, so you need to flatten the output of RNN. $\endgroup$
    – Abhishek Verma
    Apr 2, 2021 at 12:51
  • $\begingroup$ The function is simpler f(x,y) = ans. No there is no timestep. Yes I can do that with simple ANN model. I tried to do that with LSTM It worked. But still I am not sure is it correct, since I am using for the task sequential model. $\endgroup$
    – Green
    Apr 2, 2021 at 12:58
  • $\begingroup$ If your data is not sequential, you should not use LSTM or RNN. They will work but they will try to find dependencies between data that should not exist. I am not saying it is wrong. They can approximate the function. But, you are better off using 1D CNN or just Dense layers which are light on compute. In this case, it will better if you use Dense layer. Or just linear regression will work since you have a very small number of features. $\endgroup$
    – Abhishek Verma
    Apr 2, 2021 at 13:02
  • $\begingroup$ I have example of this article: sciencedirect.com/science/article/pii/S0959652619325892. It shows interpolation, yes it is made also with time. What I am trying to do is only one epizode. $\endgroup$
    – Green
    Apr 2, 2021 at 13:04
  • $\begingroup$ Ok I saw their network structure. If you trying to replicate that, then, just add flatten layer after lstm layers and then fc layer would work. They are concatenating Srw, Lat and Long data and then, feeding to the lstm layers. $\endgroup$
    – Abhishek Verma
    Apr 2, 2021 at 14:04

You must log in to answer this question.