Questions tagged [transfer-learning]

For questions related to transfer learning, a machine learning method that focuses on storing knowledge gained while solving one problem in order to apply this knowledge to a different but related problem.

Filter by
Sorted by
Tagged with
26
votes
4answers
12k views

Is it possible to train a neural network as new classes are given?

I would like to train a neural network (NN) where the output classes are not (all) defined from the start. More and more classes will be introduced later based on incoming data. This means that, every ...
6
votes
3answers
4k views

What are the differences between transfer learning and meta learning?

What are the differences between meta-learning and transfer learning? I have read 2 articles on Quora and TowardDataScience. Meta learning is a part of machine learning theory in which some ...
6
votes
2answers
2k views

What is the difference between learning without forgetting and transfer learning?

I would like to incrementally train my model with my current dataset and I asked this question on Github, which is what I'm using SSD MobileNet v1. Someone there told me about learning without ...
6
votes
2answers
109 views

What does "semantic gap" mean?

I was reading DT-LET: Deep transfer learning by exploring where to transfer, and it contains the following: It should be noted direct use of labeled source domain data on a new scene of target domain ...
5
votes
2answers
359 views

How can I train a neural network for image classification when the dataset is small?

I need to train a convolutional neural network to classify snake images. The problem is that I have only a small number of images available for some snake types. So, what is the best approach to train ...
5
votes
1answer
203 views

What are the real-life applications of transfer learning?

What are the real-life applications of transfer learning in machine learning? I am particularly interested in industrial applications of the concept.
5
votes
2answers
545 views

Why does unsupervised pre-training help in deep learning?

What is the effectiveness of pre-training of unsupervised deep learning? Does unsupervised deep learning actually work?
4
votes
1answer
2k views

What is the difference between one-shot learning, transfer learning and fine tuning?

Lately, there are lots of posts on one-shot learning. I tried to figure out what it is by reading some articles. To me, it looks like similar to transfer learning, in which we can use pre-trained ...
4
votes
1answer
77 views

What is the relation between self-taught learning and transfer learning?

I am new to transfer learning and I start by reading A Survey on Transfer Learning, and it stated the following: according to different situations of labeled and unlabeled data in the source domain, ...
4
votes
0answers
37 views

How can I improve the performance of a model trained to detect vehicle poses?

I'm looking for some suggestions on how to improve our vehicle image recognition. We have an online marketplace where customers submit photos of their vehicles. The photos need to meet certain ...
3
votes
1answer
145 views

Are there any better visual models for transfer rather than ImageNet?

Similar to the recent pushes in Pretrained Language Models (BERT, GPT2, XLNet) I was wondering if such a thrust exists in Computer Vision? From my understanding, it seems the community has converged ...
3
votes
2answers
247 views

Should you reload the optimizer for transfer learning?

For example, you train on dataset 1 with an adaptive optimizer like Adam. Should you reload the learning schedule, etc., from the end of training on dataset 1 when attempting transfer to dataset 2? ...
3
votes
2answers
111 views

Can we apply transfer learning between any two different CNN architectures?

There are many types of CNN architectures: LeNet, AlexNet, VGG, GoogLeNet, ResNet, etc. Can we apply transfer learning between any two different CNN architectures? For instance, can we apply transfer ...
3
votes
1answer
161 views

How can I detect the frame from video streaming that contains a graffiti on city wall?

I am working on a graffiti detection project. I need to analyze data stream from a camera mounted sideways on a vehicle to identify graffiti on city walls and notify authorities with the single best ...
3
votes
2answers
277 views

What is layer freezing in transfer learning?

Transfer learning consists of taking features learned on one problem and leveraging them on a new, similar problem. In the Transfer Learning, we take layers from a previously trained model and freeze ...
3
votes
1answer
69 views

Precise description of one-shot learning

I am working on classifying the Omniglot dataset, and the different papers dealing with this topic describe the problem as one-shot learning (classification). I would like to nail down a precise ...
3
votes
1answer
200 views

Transfer learning to train only for a new class while not affecting the predictions of the other class

I am basically interested in vehicle on the road. YoloV3 pytorch is giving a decent result. So my interested Vehicles Car ...
3
votes
1answer
409 views

How is transfer learning used to mitigate catastrophic forgetting in neural networks?

How can transfer learning be used to mitigate catastrophic forgetting. Could someone elaborate on this?
3
votes
1answer
329 views

When doing transfer learning, which initial layers do we need to freeze, and how should I change the last layer for my task?

I want to train a neural network for the detection of a single class, but I will be extending it to detect more classes. To solve this task, I selected the PyTorch framework. I came across transfer ...
3
votes
0answers
40 views

Why shouldn't batch normalisation layers be learnable during fine-tuning?

I have been reading this TensorFlow tutorial on transfer learning, where they unfroze the whole model and then they say: When you unfreeze a model that contains ...
3
votes
0answers
38 views

What is the difference between "out-of-distribution (generalisation)" and "(meta)-transfer learning"?

I'm trying to develop a better understanding of the concept of "out-of-distribution" (generalization) in the context of Bengio's "Moving from System 1 DL to System 2 DL" and the concept of "(meta)-...
3
votes
0answers
32 views

Training a reinforcement learning model with multiple images

I am tentatively trying to train a deep reinforcement learning model the maze escaping task, and each time it takes one image as the input (e.g., a different "maze"). Suppose I have about $10K$ ...
2
votes
4answers
199 views

Why do we have to train a model from scratch every time? [closed]

I have started on Andrew Ng's machine learning course. It seems that machine learning is learning correlations with known data based on as many parameters as possible. For example, if we collect data ...
2
votes
1answer
100 views

Is it possible to pre-train a CNN in a self-supervised way so that it can later be used to solve an instance segmentation task?

I would like to use self-supervised learning (SSL) to learn features from images (the dataset consists of similar images with small differences), then use the resulting trained model to bootstrap an ...
2
votes
1answer
51 views

Would this count as a Transfer Learning approach?

I have two datasets, Dataset 1(D1) and Dataset 2(D2). D1 has around 22000 samples, and D2 has around 8000 samples. What I am doing is that I train a Deep Neural Network model with around three layers ...
2
votes
1answer
55 views

How is few-shot learning different from transfer learning?

To my understanding, transfer learning helps to incorporate data from other related datasets and achieve the task with less labelled data (maybe in 100s of images per category). Few-shot learning ...
2
votes
1answer
56 views

Is it possible to use self-supervised learning on different images for the pretext and downstream tasks?

I have just come across the idea of self-supervised learning. It seems that it is possible to get higher accuracies on downstream tasks when the network is trained on pretext tasks. Suppose that I ...
2
votes
1answer
77 views

What are the most common methods to enable neural networks to adapt to changing environments?

For real applications, concept drifts often exist, i.e., the relationship between the input and output changes overtime. Thus, we need our AI or machine learning system to quickly adapt to the ...
2
votes
1answer
48 views

How could Bayesian neural networks be used for transfer learning?

In transfer learning, we use big data from similar tasks to learn the parameters of a neural network, and then fine-tune the neural network on our own task that has little data available for it. Here, ...
2
votes
0answers
30 views

Literature on the advantages of using an auto-encoder for classification

Given a supervised problem with X, y input pairs, one can do two things for obtaining the function f that maps X with y with Neural Networks (and in general in machine learning): Deploy directly a ...
2
votes
0answers
31 views

Is it ok to perform transfer learning with a base model for face recognition to perform one-shot learning for object classification?

I am trying to create a model that is using a one-shot learning approach for a classification task. We do this because we do not have a lot of data and it also seems like a good way to learn this ...
2
votes
0answers
475 views

Why aren't the BERT layers frozen during fine-tuning tasks?

During transfer learning in computer vision, I've seen that the layers of the base model are frozen if the images aren't too different from the model on which the base model is trained on. However, on ...
2
votes
1answer
107 views

What's the difference between domain randomization and domain adaptation?

In my understanding, domain randomization is one method of diversifying the dataset to achieve a better shot at domain adaptation. Am I wrong?
2
votes
1answer
163 views

Does self-supervised learning require auxiliary tasks?

Self-supervised learning algorithms provide labels automatically. But, it is not clear what else is required for an algorithm to fall under the category "self-supervised": Some say, self-...
2
votes
0answers
93 views

Pre-trained Models for Topic Modelling Transfer Learning (LDA) [closed]

I've been searching online - and so far, I've been unable to find any publicly-accessible pre-trained models that can be used for LDA Topic Modeling - Transfer Learning. Can anyone share any resources ...
2
votes
0answers
133 views

What is the difference between using a backbone architecture and transfer learning?

I'm super new to deep learning and computer vision, so this question may sound dumb. In this link (https://github.com/GeorgeSeif/Semantic-Segmentation-Suite), there are pre-trained models (e.g., ...
2
votes
1answer
73 views

Is my fine-tuned model learning anything at all?

I am practicing with Resnet50 fine-tuning for a binary classification task. Here is my code snippet. ...
2
votes
0answers
1k views

How to transfer learn Darknet YOLOv3

I've started getting into object detection in image. I have YOLOv3 neural network with Darknet framework. The network is pre-trained from COCO data set. Now I need to do some transfer learning in ...
2
votes
0answers
149 views

Paper & code for "unsupervised domain adaptation" for regression task

Does anyone know a paper or code that does "unsupervised domain adaptation" for regression task? I saw most of the papers were benchmarked on classification tasks, not regression. I want to do ...
1
vote
1answer
120 views

Why is my fine-tuned YOLO model detecting other objects as a human?

I am new to deep learning and computer vision. I have a problem where I use the YOLO to detect objects. For my problem, I just want to recognize 1 human only. So, I changed the final YOLO's layer (...
1
vote
0answers
9 views

Continue teaching pre-trained network without forgetting previous data set

I have a rather interesting problem here; I work in the field of image classification for quality assurance. For this I have a dataset of about 1 million images, which I have used to train different ...
1
vote
0answers
25 views

How to deal with images that do not contain any object of interest?

I'm currently working on an iOS App where I want to detect if there is a table, chair or bench in the current camera input. My idea was to take the MobileNetV2 model and get it to classify these three ...
1
vote
0answers
36 views

What is the definition of pre-training?

I want to pre-train a model (combined by two popular modules A and B, and both are large blocks), then fine-tune it on downstream tasks. What if for the weight initialization for pre-training, module ...
1
vote
0answers
16 views

Why is domain adaptation and generative modelling for knowledge graphs still not applied widely in enterprise data? What are the challenges?

I see that domain adaptation and transfer learning has been widely adopted in image classification and semantic segmentation analysis. But it's still lacking in providing solutions to enterprise data, ...
1
vote
0answers
47 views

Reference for Transfer Learning via Final Layers of a Neural Network

Problem (Sketch): I'm interested in a particular formulation of the transfer-learning problem, which, given a trained network $f$ seeks to learn a new network $g$ whose last few layers behave very ...
1
vote
3answers
134 views

How does batch normalisation actually work?

I actually went through the Keras' batch normalization tutorial and the description there puzzled me more. Here are some facts about batch normalization that I read recently and want a deep ...
1
vote
0answers
28 views

Is the high dimensionality of input vectors a problem for a radial basis function neural network?

I have a dataset A of videos. I've extracted the feature vector of each video (with a convolutional neural network, via transfer learning) creating a dataset B. Now, every vector of the dataset B has ...
1
vote
0answers
199 views

What are the actual math or computer science concepts behind these unfamiliar hyperparameters in the Deep Dream Generator's Deep Style?

I've been playing around with neural style transfer for a about a year now, and I've been doing it with two general approaches. The first has been using a script that is available on the Keras GitHub, ...
1
vote
0answers
612 views

Reasoning behind $Zero$ validation accuracy in the following ResNet50 model for classification

I have written this code to classify Cats and dogs using Resnet50. Actually while studying I came to the conclusion that Transfer learning gives very good accuracy for deep learning models, but I ...
1
vote
0answers
17 views

learning object recognition of primitive shapes through transfer learning problem

Question on transfer learning object classification (MobileNet_v2 with 75% number of parameters) with my own synthetic data: I made my own dataset of three shapes: triangles, rectangles and spheres. ...