3
$\begingroup$

In general, is continuous learning possible with a deep convolutional neural network, without changing its topology?

In my case, I want to use a convolutional neural network as a classifier of heartbeat types. The ECG signal is split, and a color image is created using feature extraction. These photos (the inputs) are fed into a deep CNN, but they must be labeled by someone first.

Are there ways to implement continuous learning in a deep neural network for image recognition? Does such an implementation make sense if the labels have to be specially prepared in advance?

$\endgroup$
  • 1
    $\begingroup$ Your question is very similar to this (which is asking about the state-of-the-art approaches, which is not your question), this (which does not seem to focus on CNNs and image recognition), this (which focuses on the case where new classes are incrementally given) or this (which asks if a CNN can be trained incrementally, without putting any restriction on the topology) questions. $\endgroup$ – nbro Sep 14 '20 at 12:29
  • 1
    $\begingroup$ Here, you are specifically asking about "incremental/continual learning for image recognition without changing the NN's topology", so I will not close this post as a duplicate of those. $\endgroup$ – nbro Nov 10 '20 at 11:32
2
$\begingroup$

In general, is continuous learning possible with a deep convolutional neural network, without changing its topology?

Your intuition that it is possible to perform incremental (aka continual, continuous or lifelong) learning by changing the NN's topology is correct. However, dynamically adapting the NN's topology is just one approach to continual learning (a specific example of this approach is DEN). So, there are other approaches, such as

For more details about these and other approaches (and problems related to continual learning and catastrophic forgetting in neural networks), take a look at this very nice review of continual learning approaches in neural networks. You should also check this answer.

Are there ways to implement continuous learning in a deep neural network for image recognition?

Yes. Many of the approaches focus on image recognition and classification, and often the experiments are performed on MNIST or similar datasets (e.g. see this paper).

Does such an implementation make sense if the labels have to be specially prepared in advance?

Yes, you can prepare your dataset in advance, and then later train incrementally (in fact, in the experiments I have seen in some of these papers, they usually do this to simulate the continual learning scenario), but I am not sure about the optimality of this approach. Maybe with batch learning (i.e. the usual offline learning where you train on all data), you would achieve higher performance.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.