I want to design a NN that can remember it's last 7 actions and use them as inputs. So for example it would be able to store words in it's memory. Therefore if it had a choice of 10 different actions, the number of words it could store is $10^7$.

Here is my design:

$$out_{n+1} = f(out_n, in_n)\mathbf{N} + out_n.\mathbf{M}$$

$$action_n = \sigma(\mathbf{N} \cdot out_n)$$

Where $f$ represents some layered neural network. Some of the actions would be physical actions and some might be internal (such as thinking of the letter 'C').

Basically I want $out_n$ to be an array that keeps the last 6 action values and puts them back in. So $M$ will be the matrix:

$$\begin{bmatrix} 0&1&0&0&0&0\\ 0&0&1&0&0&0\\ 0&0&0&1&0&0\\ 0&0&0&0&1&0\\ 0&0&0&0&0&1\\ 0&0&0&0&0&0 \end{bmatrix}$$

i.e. it would drop the 6th item from it's memory.

and $N$ would be the vector:

$$\begin{bmatrix} 1&0&0&0&0&0&0 \end{bmatrix}$$

I think this would be equivalent to an equation of the form:


So I think this would be an advantage over an RNN since this model remembers precisely it's last 6 actions. But would this be better than an RNN or worse? One could increase it's memory to more than 7 quite easily.

I think it's basically the same archececture as an RNN except elinimating a lot of the connections. Is this a new design or a common design?

One problem with this design is that you might also want a memory that is over longer time periods (e.g. for actions that take more than one tick.) But that might be solved by enhancing the archecture.


Congrats, you have invented 1d convolution. Convolution combined with RNN would have some advantage over just RNN. Think about the perception field. In this layer, you do aggregate $6$ values to one. Imagine two of them - it will be $36$ already, etc. But, in the end, you still need RNN at the end to aggregate a variable length to constant length.

  • $\begingroup$ Well that's good! Glad I'm on the right track! (Not sure what you mean at the end about variable lengths). $\endgroup$
    – zooby
    Jul 27 '19 at 19:14
  • $\begingroup$ @zooby This is not a 1D CNN, its a non differentiable RNN. (actions must be sampled under some categorical distribution based on whats described). The only similarity to a 1d cnn is the sliding window $\endgroup$
    – mshlis
    Jul 27 '19 at 19:40
  • $\begingroup$ Why is it non-differentiable ? $\endgroup$
    – zooby
    Jul 27 '19 at 19:42
  • $\begingroup$ do you train with sequences of different lenght, right? also if you put output as input think about output may be wrong so you can consider to force-feeding ( expected data instead of output) $\endgroup$ Jul 27 '19 at 20:14
  • $\begingroup$ I could be wrong but generally actions are drawn from a distribution (that’s why you show one hot encodingns) and you can’t differentiate through a categorical distrib $\endgroup$
    – mshlis
    Jul 27 '19 at 20:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.