I notice with the recent revelation of severe limitations in some AI domains such as self driving cars that NNets behave with the same sort of errors as in simpler models. Ie: They may be ~100% accuracte on test data however if you throw in a test sample that is slightly different to anything it's been trained on, it can throw the net off completely. This seems to be the case with self driving cars where NNets are miss-classifying modified/grafitied Stop Signs, unable to cope with rain or snow flakes, or birds appearing on the road, etc. Something it's never seen before in a unique climate may cause it to make completely unpredictable predictions. These specific examples may be circumvented by training on modified Stop Signs, or with rain and birds, but that avoids the point: that NN's seem very limited when it comes to generalizing to an environment that is completely unique to its training samples. And this makes sense of course given the way NNs train.

The current solution seems to be to manually find out these new things that confuse the netowrk and label them as additional training data. But that isn't an AI at all. That isn't true generalization.

I think part of the problem to blame is the term "AI" in and of itself. When all we're doing is finding a global minima to a theoretical ideal function at some perfect point before over-fitting our training data, it's obvious that the NNet cannot generalize anymore than what is possible within its training set.

I thought one way that might be possible to get around this is if rather than being static "one unique calculation at a time" if instead NNets could remember the last one or two predictions they made, and then their current prediction, and use the result of that to then make a more accurate prediction. Ie: A very basic form of short term memory.

By doing this perhaps the NN could see that the rain drops or snow flakes aren't static objects, but are simply moving noise. It could determine this by looking at its last couple of predictions and see how those objects move. Certainly this would require immense additional computation overhead, but I'm just looking to the future in terms of when processing power increases how NNs might evolve further. Similar to how NNs were already defined many decades ago but were never a "thing" due to the lack of computational power, could this be the same case with something like short term memory? That we lack the practical computational power for it but that perhaps we could theoretically implement it somehow for sometime in the future when we do have it.

Of course this short term memory thing would only be useful for when a classification is related to the prior classifications, like with self driving cars. It's important to know what was observed a few seconds ago in real life when driving, likewise it could be important for an NN to know what was predicted a few live classifications ago. This might also have use in object detection, ie: perhaps a representation could be learned for a moving figure in the distance. Speed could now become a representation in the hidden layers and used in assistance for identification of distant objects, something not possible when using a set of single static weights.

Of course this whole thing would involve somehow getting around the problem of training weights on a live model for the most recent sample. Or alternatively perhaps the weights could still remain static but we'd use two or three different models of weights to represent time somehow.

Nevertheless I can't help but see short term memory of some form as being a requirement if AI is to not be "so stupid" when it observes something unique, and if it's to ever classify things based on time and recent observations.

I'm curious if there's any research papers or other sources that explore any aspect of incorporating time using multiple recent classifications or something else that could be considered a short term memory of sorts to help reach a more general form of generalization that the NNet doesn't necessarily see in its training, ie: making it able to avoid noise using time as a feature to help it do so, or using time from multiple recent classifications as a way to estimate speed and use that as a feature? I'd appreciate the answer to include some specific experiment or methodology as to how this sort of thing might be added to NNets (or list it as a source), or if this is not an area of active research, why not.

Thank you.

  • $\begingroup$ Thank you @NeilSlater I have updated with a more specific question at the end, I hope this satisfies the requirements. $\endgroup$ – user4779 Mar 10 '19 at 9:56
  • $\begingroup$ Thanks for the edit, that looks better to me now $\endgroup$ – Neil Slater Mar 10 '19 at 10:13
  • $\begingroup$ I think that's what recurrent NNs are for; they have something that could be interpreted as a short term memory. $\endgroup$ – Oliver Mason Mar 10 '19 at 16:00

What you're describing is called a recurrent neural network. There are a large number of designs in this family that all have the ability to remember recent inputs and use them in the processing of future inputs. The "Long Short Term Memory" or LSTM architecture was one of the most successful in this family.

These are actually very widely used in things like self-driving cars too, so, on their own, they are not enough to overcome the brittleness of current models.

  • $\begingroup$ I am curious tho...Yes NN's the only thing used in cars? Or is it integrated with RL? Which among it plays the major part? $\endgroup$ – DuttaA Mar 10 '19 at 17:32
  • 1
    $\begingroup$ @DuttaA Reinforcement learning is an algorithm that provides you with an error signal for learning the relationships between states of the world, actions you could take in those states, and some measure of 'utility' or benefit to the agent. Usually this relationship is modeled as a function. One approach is to learn the function explicitly (fine for small, discrete domains). Another is to approximate the function. Neural nets are powerful function approximators, and can learn well from error signals, and this is where they fit in for the RL problem of self-driving cares. $\endgroup$ – John Doucette Mar 10 '19 at 20:19
  • 1
    $\begingroup$ You're right. I've heard of RNNs before but never delved into their specifics, now I will. Thank you. LSTM looks like an interesting way to overcome prior observations from a long time ago. $\endgroup$ – user4779 Mar 11 '19 at 4:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.