1
$\begingroup$

I want to design a NN that can remember it's last 7 actions and use them as inputs. So for example it would be able to store words in it's memory. Therefore if it had a choice of 10 different actions, the number of words it could store is $10^7$.

Here is my design:

$$out_{n+1} = f(out_n, in_n)\mathbf{N} + out_n.\mathbf{M}$$

$$action_n = \sigma(\mathbf{N} \cdot out_n)$$

Where $f$ represents some layered neural network. Some of the actions would be physical actions and some might be internal (such as thinking of the letter 'C').

Basically I want $out_n$ to be an array that keeps the last 6 action values and puts them back in. So $M$ will be the matrix:

$$\begin{bmatrix} 0&1&0&0&0&0\\ 0&0&1&0&0&0\\ 0&0&0&1&0&0\\ 0&0&0&0&1&0\\ 0&0&0&0&0&1\\ 0&0&0&0&0&0 \end{bmatrix}$$

i.e. it would drop the 6th item from it's memory.

and $N$ would be the vector:

$$\begin{bmatrix} 1&0&0&0&0&0&0 \end{bmatrix}$$

I think this would be equivalent to an equation of the form:

$$out_{n+1}=F(in_n,out_n,out_{n-1},out_{n-2},...,out_{n-6})$$

So I think this would be an advantage over an RNN since this model remembers precisely it's last 6 actions. But would this be better than an RNN or worse? One could increase it's memory to more than 7 quite easily.

I think it's basically the same archececture as an RNN except elinimating a lot of the connections. Is this a new design or a common design?

One problem with this design is that you might also want a memory that is over longer time periods (e.g. for actions that take more than one tick.) But that might be solved by enhancing the archecture.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
2
$\begingroup$

Congrats, you have invented 1d convolution. Convolution combined with RNN would have some advantage over just RNN. Think about the perception field. In this layer, you do aggregate $6$ values to one. Imagine two of them - it will be $36$ already, etc. But, in the end, you still need RNN at the end to aggregate a variable length to constant length.

Improve this answer
$\endgroup$
5
  • $\begingroup$ Well that's good! Glad I'm on the right track! (Not sure what you mean at the end about variable lengths). $\endgroup$
    – zooby
    Jul 27, 2019 at 19:14
  • $\begingroup$ @zooby This is not a 1D CNN, its a non differentiable RNN. (actions must be sampled under some categorical distribution based on whats described). The only similarity to a 1d cnn is the sliding window $\endgroup$
    – mshlis
    Jul 27, 2019 at 19:40
  • $\begingroup$ Why is it non-differentiable ? $\endgroup$
    – zooby
    Jul 27, 2019 at 19:42
  • $\begingroup$ do you train with sequences of different lenght, right? also if you put output as input think about output may be wrong so you can consider to force-feeding ( expected data instead of output) $\endgroup$
    – user8426627
    Jul 27, 2019 at 20:14
  • $\begingroup$ I could be wrong but generally actions are drawn from a distribution (that’s why you show one hot encodingns) and you can’t differentiate through a categorical distrib $\endgroup$
    – mshlis
    Jul 27, 2019 at 20:30

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .