Tag Info

Accepted

Is it possible to train a neural network as new classes are given?

I'd like to add to what's been said already that your question touches upon an important notion in machine learning called transfer learning. In practice, very few people train an entire convolutional ...
• 1,877
Accepted

What are the real-life applications of transfer learning?

One application I know of being used in industry is of image classification, by only training the last layer of one of the inception models released by Google, with the desired number of classes. I ...
• 699

Is it possible to train a neural network as new classes are given?

Here is one way you could do that. After training your network, you can save its weights to disk. This allows you to load this weights when new data becomes available and continue training pretty ...

What is the difference between learning without forgetting and transfer learning?

Learning without Forgetting (LwF) is an incremental learning (sometimes also called continual or lifelong learning) technique for neural networks, which is a machine learning technique that attempts ...
• 34.3k

What are the differences between transfer learning and meta learning?

Meta-learning is more about speeding up and optimizing hyperparameters for networks that are not trained at all, whereas transfer learning uses a net that has already been trained for some task and ...
• 317
Accepted

What is the difference between one-shot learning, transfer learning and fine tuning?

They are all related terms. From top to bottom: One-shot learning aims to achieve results with one or very few examples. Imagine an image classification task. You may show an apple and a knife to a ...
• 206
Accepted

What are the differences between transfer learning and meta learning?

First of all, I would like to say that it is possible that these terms are used inconsistently, given that at least transfer learning, AFAIK, is a relatively new expression, so, the general trick is ...
• 34.3k
Accepted

What does "semantic gap" mean?

In terms of transfer learning, semantic gap means different meanings and purposes behind the same syntax between two or more domains. For example, suppose that we have a deep learning application to ...
• 1,663
Accepted

Is there an argument against using the (reviewed) predictions of a model as ground truth to further train exactly this model?

Using the (unchecked) predictions of the model as training data is an approach known as "pseudo-labeling". It can help in certain situations, depending on the underlying structure of your ...
• 228

How can I train a neural network for image classification when the dataset is small?

Use Fine Tuning You can simply use a pre-trained model on ImageNet, as this data set has multiple snakes classes. Then you can fine tune the model with your own small data set and outputs. See this ...
• 495

Are there any better visual models for transfer rather than ImageNet?

Why is ImageNet so popular for transfer learning? Models pre-trained on the ImageNet datasets have been the de-facto choice for many years now. Many popular reasons as to why people think that ...
• 3,103
Accepted

Is it possible that the fine-tuned pre-trained model performs worse than the original pre-trained model?

Yes, this is quite the expected behavior. The main difference between expected and current behavior lies in the amount of data you are using for training VS the amount of data that the pre-trained ...
• 1,038

Is there an argument against using the (reviewed) predictions of a model as ground truth to further train exactly this model?

The answer is: It depends. What you describe is a strategy often used to save time and costs for labelling data. It is important that the data you have already labelled (the 20%) is representative of ...

What are the differences between transfer learning and meta learning?

The difference really comes down to the fact that in meta-learning, there is a population of tasks $\tau$ which have distribution $p(\tau)$. The goal is to perform well on a task drawn from $p(\tau)$. ...
• 256

Should you reload the optimizer for transfer learning?

When doing transfer learning it makes sense to have different update policies for "inherited" parameters and the "new" parameters. "Inherited" parameters are pre-trained on dataset1 and they typically ...
• 469

How can I train a neural network for image classification when the dataset is small?

Besides using transfer learning described in other answer, you should consider using siamese network. This type of network is used in cases when one does not posess many examples of objects he wants ...
• 333
Accepted

Why does unsupervised pre-training help in deep learning?

Unsupervised pre-training was done only very shortly, as far as I know, at the time when deep learning started to actually work. It extracts certain regularities in the data, which a later supervised ...
• 4,007

Why do we have to train a model from scratch every time?

Disclosure: I am a product manager on Google Cloud Platform. [...] why does everyone have to repeat the effort of learning the same things? If Google has already learned cats, or if someone ...

Is it possible to train a neural network as new classes are given?

There are several ways to add new classes to the trained model, which require just training for the new classes. Incremental training (GitHub) continuously learn a stream of data (GitHub) online ...
Accepted

How can I detect the frame from video streaming that contains a graffiti on city wall?

ResNet is an architecture for object recognition and you may use it to do your classification task. Fast RCNN may improve your results but is a more difficult architecture to implement. If you want ...
• 146

What is the difference between learning without forgetting and transfer learning?

What I want to achieve is incremental training. So, as soon as I get new data, I can further train my already trained model and I don't have to retrain everything. Learning without forgetting is one ...
• 2,551

Is it possible to pre-train a CNN in a self-supervised way so that it can later be used to solve an instance segmentation task?

Is it possible to use SSL to pre-train e.g. a faster R-CNN on a pretext task (for example, rotation), then use this pre-trained model for instance segmentation with the aim to get better accuracy? ...
• 34.3k

How is few-shot learning different from transfer learning?

They use the same techniques, but study different problems. Transfer learning always does not imply that the novel classes have very-few samples (as few as 1 per class). Few-shot learning does. The ...
Accepted

Why would the "improvement" be the result of random initialization, and so why should we use multiple runs?

Neural networks use random number generators in multiple places. Most notably for weight initialisation, but also for features such as dropout, selecting minibatches within epochs, and train/cv split ...
• 24.5k
1 vote

Can we apply transfer learning between any two different CNN architectures?

I think the answer is yes, this problem is known as "Where To transfer", please refer to this paper Learning What and Where to Transfer to know what I am talking about, and correct me if I ...
• 361
1 vote
Accepted

Can we apply transfer learning between any two different CNN architectures?

No, transfer learning cannot be applied "between" different architectures, as transfer learning is the practice of taking a neural network that has already been trained on one task and ...
• 241
1 vote
Accepted

What is layer freezing in transfer learning?

Why is this layer freezing required? It's not. What are the effects of layer freezing? The consequences are: (1) Should be faster to train (the gradient will have far less components) (2) Should ...
• 126
1 vote
Accepted

How to "forward" updated NN model to a transferred model?

I think there is no simple way to transfer knowledge changes between different models. If you take your initial model and create a new version of it which you use to learn some other task (like "...
• 705
1 vote

Precise description of one-shot learning

The model has learnt the "features" for the type of inputs, eg. faces. For the problem to be called one-shot, it needs to also correctly classify/compare any new samples. For example, in face ...
• 196
1 vote

Transfer learning to train only for a new class while not affecting the predictions of the other class

Even if you want to re-train your model for just one new class you will have to prepare your training data such that it includes all or most of the classes which you want to predict. Most of the times ...
• 111

Only top scored, non community-wiki answers of a minimum length are eligible