0
$\begingroup$

I am quite new to neural networks. I am trying to implement in Python a neural network having only one hidden layer with $N$ neurons and $1$ output layer.

The point is that I am analyzing time series and would like to use the output layer as the input of the next unit: by feeding the network with the input at time $t-1$ I obtain the output $O_{t-1}$ and, in the next step, I would like to use both the input at time $t$ and $O_{t-1}$, introducing a sort of auto-regression. I read that recurrent neural network are suitable to address this issue.

Anyway I cannot imagine how to implement a network in Keras that involves multilayer recurrence: all the references I found are linked to using the output of a layer as input of the same layer in the next step. Instead, I would like to include the output of the last layer (the output layer) in the inputs of the first hidden layer.

Improve this question
$\endgroup$
6
  • $\begingroup$ You originally tagged your post with "recurrent neural networks", so I guess you are aware of them and know what they are used for. If yes, I don't understand the purpose of this question. $\endgroup$
    – nbro
    Commented Aug 5, 2020 at 22:07
  • $\begingroup$ Sure, after a couple of days spent searching I came across "recurrent neural networks" and I have read some things about that. I just cannot understand how can such a RNN be implemented in Python. All the information I found are about networks in which the output (e.g. the hidden states) of a layer is used as input of the same layer; instead, I would like to create a recurrent structure that involves more layers. $\endgroup$
    – polemical
    Commented Aug 5, 2020 at 22:20
  • $\begingroup$ Well, you should have specified that in your question. I suggest that you edit your post to clarify that you are already aware of RNNs and what your question really is. $\endgroup$
    – nbro
    Commented Aug 5, 2020 at 22:20
  • $\begingroup$ Sorry, I think you're right. I'll do it $\endgroup$
    – polemical
    Commented Aug 5, 2020 at 22:22
  • $\begingroup$ @piotor Here's an example implementation of an LSTM, a popular type of RNN, in python: github.com/nicodjimenez/lstm/blob/master/lstm.py $\endgroup$
    – Recessive
    Commented Aug 6, 2020 at 0:14

2 Answers 2

Reset to default
0
$\begingroup$

You could just do this; concatenate your input_vector with zero's vector that has the size of your output. Then in the first pass you concatenate with the output instaid of the zero's vector. After that repeat.. At the end just compare (compute the loss) your entire output from t0 to t1 to your target and backprop.

You might want to look into recurrent layers, these are layers that have connections back to themselves so that the network can learn what to "remember". These have some problems with longer sequences, so the "newer" versions try to deal with that. (LSTM and GRU) You can also use attention mechanisms if you're dealing with sequences. (basically you learn what parts of your input sequence to look at given a certain "query", in your case maybe the last timestep) (generally used in natural language processing) But it's a bit more exotic and complicated.

Improve this answer
$\endgroup$
1
$\begingroup$

You want to look at recurrent neural networks.

Improve this answer
$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .