12 votes
Accepted

Is it possible to train a neural network as new classes are given?

I'd like to add to what's been said already that your question touches upon an important notion in machine learning called transfer learning. In practice, very few people train an entire convolutional ...
user avatar
  • 1,877
7 votes
Accepted

What are the real-life applications of transfer learning?

One application I know of being used in industry is of image classification, by only training the last layer of one of the inception models released by Google, with the desired number of classes. I ...
user avatar
  • 699
7 votes

Is it possible to train a neural network as new classes are given?

Here is one way you could do that. After training your network, you can save its weights to disk. This allows you to load this weights when new data becomes available and continue training pretty ...
user avatar
7 votes

What is the difference between learning without forgetting and transfer learning?

Learning without Forgetting (LwF) is an incremental learning (sometimes also called continual or lifelong learning) technique for neural networks, which is a machine learning technique that attempts ...
user avatar
  • 34.3k
6 votes

What are the differences between transfer learning and meta learning?

Meta-learning is more about speeding up and optimizing hyperparameters for networks that are not trained at all, whereas transfer learning uses a net that has already been trained for some task and ...
user avatar
  • 317
5 votes
Accepted

What is the difference between one-shot learning, transfer learning and fine tuning?

They are all related terms. From top to bottom: One-shot learning aims to achieve results with one or very few examples. Imagine an image classification task. You may show an apple and a knife to a ...
user avatar
  • 206
5 votes
Accepted

What are the differences between transfer learning and meta learning?

First of all, I would like to say that it is possible that these terms are used inconsistently, given that at least transfer learning, AFAIK, is a relatively new expression, so, the general trick is ...
user avatar
  • 34.3k
5 votes
Accepted

What does "semantic gap" mean?

In terms of transfer learning, semantic gap means different meanings and purposes behind the same syntax between two or more domains. For example, suppose that we have a deep learning application to ...
user avatar
  • 1,663
5 votes
Accepted

Is there an argument against using the (reviewed) predictions of a model as ground truth to further train exactly this model?

Using the (unchecked) predictions of the model as training data is an approach known as "pseudo-labeling". It can help in certain situations, depending on the underlying structure of your ...
user avatar
  • 228
4 votes

How can I train a neural network for image classification when the dataset is small?

Use Fine Tuning You can simply use a pre-trained model on ImageNet, as this data set has multiple snakes classes. Then you can fine tune the model with your own small data set and outputs. See this ...
user avatar
4 votes

Are there any better visual models for transfer rather than ImageNet?

Why is ImageNet so popular for transfer learning? Models pre-trained on the ImageNet datasets have been the de-facto choice for many years now. Many popular reasons as to why people think that ...
user avatar
  • 3,103
4 votes
Accepted

Is it possible that the fine-tuned pre-trained model performs worse than the original pre-trained model?

Yes, this is quite the expected behavior. The main difference between expected and current behavior lies in the amount of data you are using for training VS the amount of data that the pre-trained ...
user avatar
  • 1,038
4 votes

Is there an argument against using the (reviewed) predictions of a model as ground truth to further train exactly this model?

The answer is: It depends. What you describe is a strategy often used to save time and costs for labelling data. It is important that the data you have already labelled (the 20%) is representative of ...
user avatar
3 votes

What are the differences between transfer learning and meta learning?

The difference really comes down to the fact that in meta-learning, there is a population of tasks $\tau$ which have distribution $p(\tau)$. The goal is to perform well on a task drawn from $p(\tau)$. ...
user avatar
3 votes

Should you reload the optimizer for transfer learning?

When doing transfer learning it makes sense to have different update policies for "inherited" parameters and the "new" parameters. "Inherited" parameters are pre-trained on dataset1 and they typically ...
user avatar
  • 469
3 votes

How can I train a neural network for image classification when the dataset is small?

Besides using transfer learning described in other answer, you should consider using siamese network. This type of network is used in cases when one does not posess many examples of objects he wants ...
user avatar
2 votes
Accepted

Why does unsupervised pre-training help in deep learning?

Unsupervised pre-training was done only very shortly, as far as I know, at the time when deep learning started to actually work. It extracts certain regularities in the data, which a later supervised ...
user avatar
2 votes

Why do we have to train a model from scratch every time?

Disclosure: I am a product manager on Google Cloud Platform. [...] why does everyone have to repeat the effort of learning the same things? If Google has already learned cats, or if someone ...
user avatar
2 votes

Is it possible to train a neural network as new classes are given?

There are several ways to add new classes to the trained model, which require just training for the new classes. Incremental training (GitHub) continuously learn a stream of data (GitHub) online ...
user avatar
2 votes
Accepted

How can I detect the frame from video streaming that contains a graffiti on city wall?

ResNet is an architecture for object recognition and you may use it to do your classification task. Fast RCNN may improve your results but is a more difficult architecture to implement. If you want ...
user avatar
  • 146
2 votes

What is the difference between learning without forgetting and transfer learning?

What I want to achieve is incremental training. So, as soon as I get new data, I can further train my already trained model and I don't have to retrain everything. Learning without forgetting is one ...
user avatar
  • 2,551
2 votes

Is it possible to pre-train a CNN in a self-supervised way so that it can later be used to solve an instance segmentation task?

Is it possible to use SSL to pre-train e.g. a faster R-CNN on a pretext task (for example, rotation), then use this pre-trained model for instance segmentation with the aim to get better accuracy? ...
user avatar
  • 34.3k
2 votes

How is few-shot learning different from transfer learning?

They use the same techniques, but study different problems. Transfer learning always does not imply that the novel classes have very-few samples (as few as 1 per class). Few-shot learning does. The ...
user avatar
2 votes
Accepted

Why would the "improvement" be the result of random initialization, and so why should we use multiple runs?

Neural networks use random number generators in multiple places. Most notably for weight initialisation, but also for features such as dropout, selecting minibatches within epochs, and train/cv split ...
user avatar
  • 24.5k
1 vote

Can we apply transfer learning between any two different CNN architectures?

I think the answer is yes, this problem is known as "Where To transfer", please refer to this paper Learning What and Where to Transfer to know what I am talking about, and correct me if I ...
user avatar
1 vote
Accepted

Can we apply transfer learning between any two different CNN architectures?

No, transfer learning cannot be applied "between" different architectures, as transfer learning is the practice of taking a neural network that has already been trained on one task and ...
user avatar
1 vote
Accepted

What is layer freezing in transfer learning?

Why is this layer freezing required? It's not. What are the effects of layer freezing? The consequences are: (1) Should be faster to train (the gradient will have far less components) (2) Should ...
user avatar
  • 126
1 vote
Accepted

How to "forward" updated NN model to a transferred model?

I think there is no simple way to transfer knowledge changes between different models. If you take your initial model and create a new version of it which you use to learn some other task (like "...
user avatar
  • 705
1 vote

Precise description of one-shot learning

The model has learnt the "features" for the type of inputs, eg. faces. For the problem to be called one-shot, it needs to also correctly classify/compare any new samples. For example, in face ...
user avatar
1 vote

Transfer learning to train only for a new class while not affecting the predictions of the other class

Even if you want to re-train your model for just one new class you will have to prepare your training data such that it includes all or most of the classes which you want to predict. Most of the times ...
user avatar

Only top scored, non community-wiki answers of a minimum length are eligible