63

Yes, indeed, neural networks are very prone to catastrophic forgetting (or interference). Currently, this problem is often ignored because neural networks are mainly trained offline (sometimes called batch training), where this problem does not often arise, and not online or incrementally, which is fundamental to the development of artificial general ...


21

Yes, the problem of forgetting older training examples is a characteristic of Neural Networks. I wouldn't call it a "flaw" though because it helps them be more adaptive and allows for interesting applications such as transfer learning (if a network remembered old training too well, fine tuning it to new data would be meaningless). In practice what you want ...


11

I'd like to add to what's been said already that your question touches upon an important notion in machine learning called transfer learning. In practice, very few people train an entire convolutional network from scratch (with random initialization), because it is time consuming and relatively rare to have a dataset of sufficient size. Modern ConvNets ...


6

There are several terms or expressions related to such systems, such as online learning, incremental learning, continuous learning, continual learning, and lifelong learning. They are sometimes used interchangeably, but some of them have slightly different meanings. For example, online learning does not need to be incremental, which refers to algorithms that ...


6

Here is one way you could do that. After training your network, you can save its weights to disk. This allows you to load this weights when new data becomes available and continue training pretty much from where your last training left off. However, since this new data might come with additional classes, you now do pre-training or fine-tuning on the network ...


6

Learning without Forgetting (LwF) is an incremental learning (sometimes also called continual or lifelong learning) technique for neural networks, which is a machine learning technique that attempts to avoid catastrophic forgetting. There are several incremental learning approaches. LwF is an incremental learning approach based on the concept of ...


5

What you are describing sounds like it could be a deliberate case of fine-tuning. There is a fundamental assumption that makes minibatch gradient descent work for learning problems: It is assumed that any batch or temporal window of consecutive batches forms a decent approximation of the true global gradient of the error function with respect to any ...


4

This article on Dynamically Expandable Neural Networks (DEN) (by Harshvardhan Gupta) is based on this paper Lifelong Learning with Dynamically Expandable Networks (by Jeongtae Lee, Jaehong Yoon, Eunho Yang, Sung Ju Hwang) This presents 3 solutions to increase the capacity of the network if needed retaining whatever useful information from the old model and ...


4

You are looking for incremental (or online) learning. A CNN can be trained incrementally. For example, in the paper Incremental Learning of Convolutional Neural Networks, the authors propose an incremental learning algorithm (inspired by AdaBoost and Learn++, which is another incremental learning algorithm for supervised learning of neural networks) for ...


3

Do you know which are the state-of-the-art approaches on this topic, and could you point me to some literature on them? This answer already mentions some of the approaches. More concretely, currently, the most common approaches to continual learning (i.e. learning with progressively more data while attempting to address the catastrophic forgetting problem) ...


3

Yes, this is possible. Continuously extending your training data is known as incremental learning. You might also want to take a look at transfer learning, in which you reuse a trained model for a different purpose. This is very useful if you have a smaller dataset. In your particular case, you could train a NN once using your data from 2010 to 2019 and ...


3

I mostly studied HMMs and such models are called Infinite HMMs in that specific domain. I believe that what you are looking for is called Infinite Neural Networks. Not having access to scientific publications, I cannot really refer any work here. However, I found this GitHub repository: https://github.com/kutoga/going_deeper that provides some ...


2

First, the title mentions "sparse data". Recently the expression has taken a clear meaning: The agent input is data with mostly zeros. In the question a different meaning: A "sparse data stream", where data flows and vanishes sometimes. I understand the question as: "Will training an AI still work if the training data stream breaks?" Note the explicit "...


2

Near solution to your problem definition is reinforcement learning. You can define some reward using the objective function and define some possible state space for the machine and finally solve the problem by reinforcement learning techniques (near to trial and error by learning the preferences).


2

What I want to achieve is incremental training. So, as soon as I get new data, I can further train my already trained model and I don't have to retrain everything. Learning without forgetting is one of the methods to solve multitask learning. If your model trained to solve problem A and then after sometimes you need your model to solve new problem B without ...


2

What I understand from your questions is that you are trying to avoid catastrophic forgetting while applying online learning. This problem should be addressed by implementing methods that reduce catastrophic forgetting for different tasks. At first glance it might seem that they don't apply because it's data that change and not a particular task but ...


2

In general, is continuous learning possible with a deep convolutional neural network, without changing its topology? Your intuition that it is possible to perform incremental (aka continual, continuous or lifelong) learning by changing the NN's topology is correct. However, dynamically adapting the NN's topology is just one approach to continual learning (a ...


2

There are lots of different approaches that try to avoid catastrophic forgetting in neural networks. It is impossible to summarize all contributions here. However, in addition to the already mentioned techniques, there are sparsity approaches that try to disentangle internal representations of the network on different tasks or learning steps. Sparsity ...


1

That is exactly a neural network works like. Suppose you have a 1000 examples. How you train a network is: First, you divide these 1000 into maybe 100 batches (10 each). After that's done, you feed a batch to the network get its output and compare it with the ground truth, whatever is the error gets backpropagated. Then, for the next batch and then another. ...


1

You are right. If you don't continuously train the neural network after you have deployed it, there is no way it can continuously learn or be updated with more information. You need to program the neural network to learn even after it has been deployed. There is no such thing as a neural network that decides what it does without a human deciding first what ...


1

You are probably looking for incremental learning (sometimes known as lifelong learning) techniques, i.e. machine learning techniques that attempt to address the catastrophic forgetting effect of neural networks when trained incrementally, i.e. as new classes or data are added to the original training data. There are different techniques and some of them ...


1

Even if you want to re-train your model for just one new class you will have to prepare your training data such that it includes all or most of the classes which you want to predict. Most of the times last two layers of a network have the data of number of labels which are to be predicted and that should always be sum of the number of classes you already ...


1

For the vast majority of cases where you have a dynamic(and assumed non-linear) relationship between your input and output, you would not use modified architecture. You would simply retrain on the new data. In some cases, based on domain knowledge or intuition, one might put a "weight" on the new data to increase or decrease its importance relative to ...


1

Maybe in theory, but not in practice. The thing is you seem to consider only chronological/sequential training. And there are two ways to view this issue: online learning -> then it is a feature of the method offline learning -> it does not happen thanks to several order randomizations 1. Online-Training or Online Machine Learning. Using the woppal ...


1

I believe "Reinforcement Learning" is the term you are looking for (as mentioned by others as well) but keep in mind that the scope of your problem falls under the section of AI that is called Search. Search algorithms are based upon experimenting with different actions (decisions) and selecting the one that minimizes an arbitrary cost function (reward), ...


1

I think any learning algorithm probably uses trial and error and analysis of the results with the ultimate goal of maximizing utility. It seems that the recent milestones in AI fall under the general umbrella of machine learning, which includes all forms of reinforcement learning. Essentially, any learning algorithm is using some form of statistical ...


1

There are several ways to add new classes to the trained model, which require just training for the new classes. Incremental training (GitHub) continuously learn a stream of data (GitHub) online machine learning (GitHub) Transfer Learning Twice Continual learning approaches (Regularization, Expansion, Rehearsal) (GitHub)


1

ML, being a relatively young and fast-developing field, has numerous (near-)synonyms for many concepts. One paradigm difference is whether a model is learned from a static, pre-defined set of data, or whether it adapts as new data is presented to it over time. Some of the terms used to describe these two paradigms respectively (with subtle differences in ...


1

In such a thriving field like AI, many terms are not fully established and in some areas it will take more time until the community agrees on specific terms for specific technologies. Therefore, it might not be possible to give an definitive answer to your question yet. But it is important to note, that the references you gave do not all speak about the ...


Only top voted, non community-wiki answers of a minimum length are eligible