I have a time-varying input size vector for a RNN. However, I am facing some difficulties understanding how to deal with my network weights when the input changes.

Say we have a set of natural positive integers $$ \Gamma=\{1,2,\dots,F\}, $$ where $F=100$ for the sake of the example.

A valid observation vector of my agent at time $t$ might be $$ \gamma_t=[1,3,5,1]. $$ Thus, at time $t$ a set of weights will be produced by my RNN, according to $\gamma_t$. Sat that at time $t+1$, my observation vector changes as $$ \gamma_{t+1}=[3,5,8], $$ and there is my problem. If I now continue training my RNN with the previous weights, the output would be inevitably affected. Also, Which weight shall I remove? I see RNN can face the issue but how shall I deal with the previously computed weights? Which one to remove? How to initialize a new one in case the cardinality of $\gamma_{t+1}$ is higher than that of $\gamma_t$?

  • $\begingroup$ is each $\gamma_t$ a new input to the RNN? or is the temporal component that it feeds in all $\gamma$'s at once? Also what weight are you referring to, when asking about "removing"? $\endgroup$ – mshlis Jun 11 '19 at 12:18
  • $\begingroup$ $\gamma_t$ is the input of the RNN at time $t$. Let's say, $\gamma_t$ is a vector of requests that have to be satisfied by the agent. It might be that a request (num. $1$ in my example) is satisfied earlier and so the vector $\gamma_t$ will not present that element at time $t+1$. At the same time, a new request comes in (num. $8$ in my example) and so $\gamma_{t+1}$ will present this new value. $\endgroup$ – EmG Jun 11 '19 at 14:51
  • $\begingroup$ About the weights, I'm wondering if this change on the input layer would create some confusion to the RNN which will have a set of weights for the linear combination that refer to the previous condition. Have I been clear for what I intend as 'weights'? Thanks for your answer $\endgroup$ – EmG Jun 11 '19 at 14:51
  • $\begingroup$ i see, so youre problem is like one long temporal vector, but for the rnn you only let it see parts that fall into some form of bin (where bins overlap?). Can you clarify your problem a bit more specifically? $\endgroup$ – mshlis Jun 11 '19 at 15:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.