There seems to be a lot of literature and research on the problems of stochastic gradient descent and catastrophic forgetting, but I can't find much on solutions to perform online learning with neural network architectures.

By online learning, I mean improving a model (while using it) with a stream of data coming in (maybe after a partial initial training with ordinary batches and epochs).

A lot of real-world distributions are likely to gradually change with time, so I believe that we should be able to train NNs in an online fashion.

Do you know which are the state-of-the-art approaches on this topic, and could you point me to some literature on them?


What I understand from your questions is that you are trying to avoid catastrophic forgetting while applying online learning.

This problem should be addressed by implementing methods that reduce catastrophic forgetting for different tasks. At first glance it might seem that they don't apply because it's data that change and not a particular task but changing data result in a change of the task. Say your goal is to classify different breeds of dogs. Your online data-set morphs into excluding "Great Danes". Your neural network after enough epochs would forget about "Great Danes". The task is still serving its purpose by classifying different breeds but the task still changed. It changed from recognizing "Great Danes" as a dog breed to not recognizing "Great Danes" as a dog breed. The weights changed to exclude them but the methods I linked tries and keep weights intact even though it was not intended for the purpose of online learning. Just set the hyper parameters to include these techniques to low as I believe data won't have an instant change but would change over time, and you should be fine.

The most obvious technique being storing information as you train. This is called pseudo-rehearsal. With this at least you would be able to use stochastic gradient decent but you need memory and resources as the data set grows.

Then there was an attempt to reduce impacts of weights on old tasks to keep some relevancy to them. Structural Regularization.

Later these guys implemented HAT which seems to keep some weights static while others adapt to new tasks.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.