What are the state-of-the-art approaches for continual learning with neural networks?

Question

There seems to be a lot of literature and research on the problems of stochastic gradient descent and catastrophic forgetting, but I can't find much on solutions to perform continual learning with neural network architectures.

By continual learning, I mean improving a model (while using it) with a stream of data coming in (maybe after a partial initial training with ordinary batches and epochs).

A lot of real-world distributions are likely to gradually change with time, so I believe that we should be able to train NNs in an online fashion.

Do you know which are the state-of-the-art approaches on this topic, and could you point me to some literature on them?

Answer 1

What I understand from your questions is that you are trying to avoid catastrophic forgetting while applying online learning.

This problem should be addressed by implementing methods that reduce catastrophic forgetting for different tasks. At first glance it might seem that they don't apply because it's data that change and not a particular task but changing data result in a change of the task. Say your goal is to classify different breeds of dogs. Your online data-set morphs into excluding "Great Danes". Your neural network after enough epochs would forget about "Great Danes". The task is still serving its purpose by classifying different breeds but the task still changed. It changed from recognizing "Great Danes" as a dog breed to not recognizing "Great Danes" as a dog breed. The weights changed to exclude them but the methods I linked tries and keep weights intact even though it was not intended for the purpose of online learning. Just set the hyper parameters to include these techniques to low as I believe data won't have an instant change but would change over time, and you should be fine.

The most obvious technique being storing information as you train. This is called pseudo-rehearsal. With this at least you would be able to use stochastic gradient decent but you need memory and resources as the data set grows.

Then there was an attempt to reduce impacts of weights on old tasks to keep some relevancy to them. Structural Regularization.

Later these guys implemented HAT which seems to keep some weights static while others adapt to new tasks.

Answer 2

Do you know which are the state-of-the-art approaches on this topic, and could you point me to some literature on them?

This answer already mentions some of the approaches. More concretely, currently, the most common approaches to continual learning (i.e. learning with progressively more data while attempting to address the catastrophic forgetting problem) are

• dynamic/changing topologies approaches
• regularization approaches
• rehearsal (or pseudo-rehearsal) approaches
• ensemble approaches
• hybrid approaches

You can also take a look at this answer. If you are interested in an exhaustive overview of the state-of-the-art (at least, until 2019), you should read the paper Continual lifelong learning with neural networks: A review (2019, by Parisi et al.).

Answer 3

There are lots of different approaches that try to avoid catastrophic forgetting in neural networks. It is impossible to summarize all contributions here.

However, in addition to the already mentioned techniques, there are sparsity approaches that try to disentangle internal representations of the network on different tasks or learning steps. Sparsity usually helps, but the network has to learn to use it, imposing a structural sparsity by construction is not enough. Also, you can leverage bayesian approaches, through which you can associate a confidence measure to each of your weights and use this measure to mitigate forgetting. Also, meta-learning can be employed to meta-learn a model which is robust to forgetting on different sequences of tasks.

What I can suggest you in addition, is to take a look at ContinualAI wiki, which maintains a list of updated publications classified by the type of Continual Learning strategy and tagged with additional information. (Disclaimer: I am a member of ContinualAI association).