Questions tagged [transfer-learning]
For questions related to transfer learning, a machine learning method that focuses on storing knowledge gained while solving one problem in order to apply this knowledge to a different but related problem.
77
questions
0
votes
1
answer
52
views
Are foundation models something fundamentally new? Is there a proper definition?
Currently, one can hear more and more about "foundation models" but details of this are not always clear. Also, I even have the impression, that sometimes people don't talk about exactly ...
- 156
0
votes
0
answers
13
views
What are some good pairs of transfer learning source and target datasets for image classification?
As the title says, I'm curious about some well used transfer learning tasks.
ImageNet to other datasets is common, but what are something good pairs I can try and mess around with ?
Like CIFAR10 to ...
- 11
0
votes
0
answers
32
views
Can we naively merge source and target datasets to train for the same task instead of performing domain adaptation?
I have seen from literature that models such as DANN or ADDA are typical in the field of Domain Adaptation, a branch of transductive learning. I know that these methods are extremely useful especially ...
- 1
0
votes
0
answers
17
views
Approximate weight matrices of pretrained models
I am looking for a guide on matrix approximation of pretrained models.
My idea is related to transfer learning: I want to use a pretrained model, take the weights and biases of one of its layers, ...
0
votes
1
answer
27
views
Self Supervised Learning Application of trained model... A bit confused
I am trying to apply a self supervised task as stated in this github repo.The Self-Supervised Sketch Recognition
In this work, authors are using 345.000 image samples to train the model and the ...
- 1
0
votes
0
answers
22
views
Does splitting the classes in my dataset into sub classes improve classification accuracy?
My problem is basically classifying ok / not ok. But I do have additional information on the error cases for the "not ok" class. Should I just train on the classes that I need for my output, ...
- 73
0
votes
0
answers
53
views
Can I use a pre-trained BERT to generate embeddings for training dataset then to fine tune the same BERT for semantic similarity?
I would like to fine-tune a sentence BERT model using my own dataset and perform a semantic similarity task. When generating the training dataset, I need to generate the embeddings for each sentence ...
- 101
1
vote
2
answers
466
views
Does layer freezing offer other benefits other than to reduce computational time in gradient descent?
In Deep Learning and Transfer Learning, does layer freezing offer other benefits other than to reduce computational time in gradient descent?
Assuming I train a neural network on task A to derive ...
- 21
0
votes
0
answers
30
views
What is a good code-base for image classification using transfer learning?
I want to train a model for an image classification task. I want to use a pretrained model, e.g. ResNet50 trained on imagenet.
Is there a good, easy-to-use code base for training the model and ...
- 73
0
votes
0
answers
50
views
Transfer learning (or fine-tuning) a pre-trained model on multiple features?
I am currently fine-tuning a sentiment analysis bert-based model using PyTorch Trainer from hugging face. So far, so good.
I have easily managed to fine-tune the model on my text data. However, I'd ...
- 101
0
votes
0
answers
25
views
How is model training affected after randomizing the weights of an intermediate layer of a pre-trained model?
Assuming that I have a deep learning model (let's say a ResNet) pretrained on a given dataset (let's say it is ImageNet). I load that model and randomize the weights of one of the intermediate layers, ...
0
votes
1
answer
40
views
Keep weights of output layer in transfer learning?
I'm seeing conflicting info on what to do with the fully-connected output layer of a pre-trained network when it's used in transfer learning. A previous answer seems to imply that the network is kept ...
2
votes
1
answer
65
views
Is transfer learning effective when the new task has more classes than the original?
All examples of transfer learning I have seen for classification use initial weights of a network trained on a larger number of classes (say 1000 in the case of networks trained on ImageNet data) to ...
1
vote
1
answer
106
views
Embedding Quality of Transfer Learning model vs Contrastive learning model
I am working on Contrastive learning which is a technique to learn features based on the concept of learning from comparing two or more instances.
The downstream task is a classification problem.
...
7
votes
2
answers
790
views
Is there an argument against using the (reviewed) predictions of a model as ground truth to further train exactly this model?
I plan to use my predictions as ground truth to continue training my model. These predictions are of course reviewed during this process. Is there an argument against that (reinforcement of slight ...
- 73
2
votes
0
answers
60
views
How to choose the new layer and objective function for transfer learning on a neural network?
I have a base model $M$ trained on a data say type 1 for task $T$. Now, I want to update $M$ by applying transfer learning for it to work on data type 2 for the same task $T$. I am very new to AI/ML ...
- 121
1
vote
0
answers
25
views
Transferring a Q-learning policy to larger instances
How do I best transfer and fine-tune a Q-learning policy that was trained on small instances to large instances?
Some more details on the problem:
I am currently trying to derive a decision policy for ...
1
vote
0
answers
13
views
Continue teaching pre-trained network without forgetting previous data set
I have a rather interesting problem here; I work in the field of image classification for quality assurance.
For this I have a dataset of about 1 million images, which I have used to train different ...
- 131
0
votes
1
answer
46
views
Should I label static objects on video dataset?
I'm using nvidia Transfer Learning Toolkit to detect cars in some video frames.
I found some dataset (for example https://www.jpjodoin.com/urbantracker/dataset.html and https://www.kaggle.com/...
0
votes
1
answer
1k
views
Is it possible that the fine-tuned pre-trained model performs worse than the original pre-trained model?
I have downloaded a pre-trained EfficientDet D2 model (Tensorflow 2.0) and trained it on some data (about 20000 images with 20 classes). I set the number of steps to 25000 and batch size to 3 (...
- 103
0
votes
0
answers
55
views
Model not learning anything, what can be the problem?
I've trained a model for heart sound classification with transfer learning (MobileNet) on Physionet dataset, and it works fine.
However, when I train it on my own dataset, it seems that it can not ...
2
votes
1
answer
365
views
How could Bayesian neural networks be used for transfer learning?
In transfer learning, we use big data from similar tasks to learn the parameters of a neural network, and then fine-tune the neural network on our own task that has little data available for it. Here, ...
0
votes
1
answer
90
views
Why would the "improvement" be the result of random initialization, and so why should we use multiple runs?
I got this feedback for my thesis paper.
The improvement shown in the results section could be the result of random initialization. There should be multiple runs with means and standard deviations.
...
1
vote
0
answers
129
views
How to deal with images that do not contain any object of interest?
I'm currently working on an iOS App where I want to detect if there is a table, chair or bench in the current camera input.
My idea was to take the MobileNetV2 model and get it to classify these three ...
- 11
1
vote
1
answer
98
views
Would this count as a Transfer Learning approach?
I have two datasets, Dataset 1(D1) and Dataset 2(D2). D1 has around 22000 samples, and D2 has around 8000 samples. What I am doing is that I train a Deep Neural Network model with around three layers ...
- 133
0
votes
1
answer
597
views
Validation accuracy very low with transfer learning
I am using MobileNetV3 from TF keras for doing transfer learning; I removed the last layer, added two dense layers, and trained for 20 epochs.
How many dense layers should I add after the MobileNet ...
2
votes
2
answers
3k
views
What is the difference between feature extraction and fine-tuning in transfer learning?
I'm building a model for facial expression recognition, and I want to use transfer learning. From what I understand, there are different steps to do it. The first is the feature extraction and the ...
- 21
0
votes
1
answer
117
views
How to perform multi-class text classification with a dataset of 80 documents?
I have a training dataset of 80 text documents with an average number of characters in each document of 25000 and 210 unique tags.
How can I perform multi-class text classification with such a small ...
-1
votes
1
answer
131
views
How to train my model using transfer learning on inception_v3 pre-trained model?
I am trying to train my model to classify 10 classes of hand gestures but I don't get why am I getting validation accuracy approx. double than training accuracy.
My dataset is from kaggle:
https://www....
6
votes
2
answers
1k
views
What does "semantic gap" mean?
I was reading DT-LET: Deep transfer learning by exploring where to transfer, and it contains the following:
It should be noted direct use of labeled source domain data on a new scene of target domain ...
- 361
4
votes
1
answer
180
views
What is the relation between self-taught learning and transfer learning?
I am new to transfer learning and I start by reading A Survey on Transfer Learning, and it stated the following:
according to different situations of labeled and unlabeled data in the source domain, ...
- 361
1
vote
0
answers
53
views
What is the definition of pre-training?
I want to pre-train a model (combined by two popular modules A and B, and both are large blocks), then fine-tune it on downstream tasks.
What if for the weight initialization for pre-training, module ...
- 11
4
votes
1
answer
3k
views
How is few-shot learning different from transfer learning?
To my understanding, transfer learning helps to incorporate data from other related datasets and achieve the task with less labelled data (maybe in 100s of images per category).
Few-shot learning ...
- 61
3
votes
0
answers
118
views
Why shouldn't batch normalisation layers be learnable during fine-tuning?
I have been reading this TensorFlow tutorial on transfer learning, where they unfroze the whole model and then they say:
When you unfreeze a model that contains ...
- 862
2
votes
0
answers
36
views
Literature on the advantages of using an auto-encoder for classification
Given a supervised problem with X, y input pairs, one can do two things for obtaining the function f that maps X with y with Neural Networks (and in general in machine learning):
Deploy directly a ...
2
votes
1
answer
123
views
Is it possible to use self-supervised learning on different images for the pretext and downstream tasks?
I have just come across the idea of self-supervised learning. It seems that it is possible to get higher accuracies on downstream tasks when the network is trained on pretext tasks.
Suppose that I ...
- 1,251
2
votes
1
answer
288
views
Is it possible to pre-train a CNN in a self-supervised way so that it can later be used to solve an instance segmentation task?
I would like to use self-supervised learning (SSL) to learn features from images (the dataset consists of similar images with small differences), then use the resulting trained model to bootstrap an ...
- 21
0
votes
1
answer
419
views
Transfer Learning of Numerical Data
It seems like transfer learning is only applicable to neural networks. Is this a correct assumption?
While I was looking for examples of Transfer Learning, most seemed to be based on image data, audio ...
2
votes
0
answers
61
views
Is it ok to perform transfer learning with a base model for face recognition to perform one-shot learning for object classification?
I am trying to create a model that is using a one-shot learning approach for a classification task. We do this because we do not have a lot of data and it also seems like a good way to learn this ...
- 141
4
votes
1
answer
2k
views
Why aren't the BERT layers frozen during fine-tuning tasks?
During transfer learning in computer vision, I've seen that the layers of the base model are frozen if the images aren't too different from the model on which the base model is trained on.
However, on ...
- 141
3
votes
2
answers
460
views
Can we apply transfer learning between any two different CNN architectures?
There are many types of CNN architectures: LeNet, AlexNet, VGG, GoogLeNet, ResNet, etc. Can we apply transfer learning between any two different CNN architectures? For instance, can we apply transfer ...
- 547
0
votes
1
answer
63
views
BERT: After pretraining 880000 step, why fine-tune not work? [closed]
I am using pretraining code from https://github.com/NVIDIA/DeepLearningExamples
Pretrain parameters:
...
- 101
1
vote
0
answers
24
views
Why is domain adaptation and generative modelling for knowledge graphs still not applied widely in enterprise data? What are the challenges?
I see that domain adaptation and transfer learning has been widely adopted in image classification and semantic segmentation analysis. But it's still lacking in providing solutions to enterprise data, ...
- 11
3
votes
1
answer
506
views
What's the difference between domain randomization and domain adaptation?
In my understanding, domain randomization is one method of diversifying the dataset to achieve a better shot at domain adaptation. Am I wrong?
- 131
4
votes
2
answers
2k
views
What is layer freezing in transfer learning?
Transfer learning consists of taking features learned on one problem and leveraging them on a new, similar problem.
In the Transfer Learning, we take layers from a previously trained model and freeze ...
- 1,243
1
vote
3
answers
211
views
How does batch normalisation actually work?
I actually went through the Keras' batch normalization tutorial and the description there puzzled me more.
Here are some facts about batch normalization that I read recently and want a deep ...
0
votes
1
answer
107
views
How to "forward" updated NN model to a transferred model?
I've trained a robot to walk in a straight line for as long as it can (using TD3), and now I'm using that pre-trained model for two new models with separate purposes: 1. Walk to a specific point and ...
- 35
3
votes
1
answer
313
views
Does self-supervised learning require auxiliary tasks?
Self-supervised learning algorithms provide labels automatically. But, it is not clear what else is required for an algorithm to fall under the category "self-supervised":
Some say, self-...
- 163
12
votes
1
answer
4k
views
What is the difference between one-shot learning, transfer learning and fine tuning?
Lately, there are lots of posts on one-shot learning. I tried to figure out what it is by reading some articles. To me, it looks like similar to transfer learning, in which we can use pre-trained ...
- 443
1
vote
0
answers
61
views
Is the high dimensionality of input vectors a problem for a radial basis function neural network?
I have a dataset A of videos. I've extracted the feature vector of each video (with a convolutional neural network, via transfer learning) creating a dataset B. Now, every vector of the dataset B has ...
- 167