Skip to main content

Questions tagged [transfer-learning]

For questions related to transfer learning, a machine learning method that focuses on storing knowledge gained while solving one problem in order to apply this knowledge to a different but related problem.

Filter by
Sorted by
Tagged with
1 vote
0 answers
61 views

AI chat bot that answers by focusing only on 30 textbooks [closed]

I don't even know what I'm looking for and what's the terminology, so here I am asking this question. Background Assume I have 30 textbooks. I want to have an AI chatbot like ChatGPT which answers the ...
Megidd's user avatar
  • 141
1 vote
0 answers
47 views

How to fine-tune pre-trained model? [closed]

I'm trying to classify a data set of medical images with a pre-trained model EfficientNetB0. I've written a code in Python with Pytorch to train my model and fine-tune it but I would like to know if ...
NitaStack's user avatar
1 vote
0 answers
63 views

SOTA model (PAtt-lite) implementation with Pytorch [closed]

I'm trying to implement a model guided by the paper of PAtt-lite (weights are on https://github.com/jlrex/patt-lite but no implementation provided yet). Using FER2013+ and RAF-DB. The main classes I'...
Robert's user avatar
  • 111
1 vote
1 answer
80 views

Are foundation models something fundamentally new? Is there a proper definition?

Currently, one can hear more and more about "foundation models" but details of this are not always clear. Also, I even have the impression, that sometimes people don't talk about exactly ...
BanDoP's user avatar
  • 193
0 votes
0 answers
13 views

What are some good pairs of transfer learning source and target datasets for image classification?

As the title says, I'm curious about some well used transfer learning tasks. ImageNet to other datasets is common, but what are something good pairs I can try and mess around with ? Like CIFAR10 to ...
v1998199904's user avatar
0 votes
0 answers
35 views

Can we naively merge source and target datasets to train for the same task instead of performing domain adaptation?

I have seen from literature that models such as DANN or ADDA are typical in the field of Domain Adaptation, a branch of transductive learning. I know that these methods are extremely useful especially ...
Haneul's user avatar
  • 1
0 votes
0 answers
19 views

Approximate weight matrices of pretrained models

I am looking for a guide on matrix approximation of pretrained models. My idea is related to transfer learning: I want to use a pretrained model, take the weights and biases of one of its layers, ...
postnubilaphoebus's user avatar
0 votes
1 answer
34 views

Self Supervised Learning Application of trained model... A bit confused

I am trying to apply a self supervised task as stated in this github repo.The Self-Supervised Sketch Recognition In this work, authors are using 345.000 image samples to train the model and the ...
T.Gulez's user avatar
1 vote
2 answers
620 views

Does layer freezing offer other benefits other than to reduce computational time in gradient descent?

In Deep Learning and Transfer Learning, does layer freezing offer other benefits other than to reduce computational time in gradient descent? Assuming I train a neural network on task A to derive ...
Enk9456's user avatar
  • 21
0 votes
1 answer
43 views

Keep weights of output layer in transfer learning?

I'm seeing conflicting info on what to do with the fully-connected output layer of a pre-trained network when it's used in transfer learning. A previous answer seems to imply that the network is kept ...
Fijoy Vadakkumpadan's user avatar
2 votes
1 answer
88 views

Is transfer learning effective when the new task has more classes than the original?

All examples of transfer learning I have seen for classification use initial weights of a network trained on a larger number of classes (say 1000 in the case of networks trained on ImageNet data) to ...
Fijoy Vadakkumpadan's user avatar
1 vote
1 answer
122 views

Embedding Quality of Transfer Learning model vs Contrastive learning model

I am working on Contrastive learning which is a technique to learn features based on the concept of learning from comparing two or more instances. The downstream task is a classification problem. ...
Raj Rajeshwari Prasad's user avatar
7 votes
2 answers
798 views

Is there an argument against using the (reviewed) predictions of a model as ground truth to further train exactly this model?

I plan to use my predictions as ground truth to continue training my model. These predictions are of course reviewed during this process. Is there an argument against that (reinforcement of slight ...
thzu's user avatar
  • 73
2 votes
0 answers
61 views

How to choose the new layer and objective function for transfer learning on a neural network?

I have a base model $M$ trained on a data say type 1 for task $T$. Now, I want to update $M$ by applying transfer learning for it to work on data type 2 for the same task $T$. I am very new to AI/ML ...
PHcoDer's user avatar
  • 121
1 vote
0 answers
25 views

Transferring a Q-learning policy to larger instances

How do I best transfer and fine-tune a Q-learning policy that was trained on small instances to large instances? Some more details on the problem: I am currently trying to derive a decision policy for ...
BotsAgainstCaptchas's user avatar
1 vote
0 answers
14 views

Continue teaching pre-trained network without forgetting previous data set

I have a rather interesting problem here; I work in the field of image classification for quality assurance. For this I have a dataset of about 1 million images, which I have used to train different ...
beinando's user avatar
  • 131
0 votes
1 answer
47 views

Should I label static objects on video dataset?

I'm using nvidia Transfer Learning Toolkit to detect cars in some video frames. I found some dataset (for example https://www.jpjodoin.com/urbantracker/dataset.html and https://www.kaggle.com/...
Francesco Pagani's user avatar
0 votes
1 answer
1k views

Is it possible that the fine-tuned pre-trained model performs worse than the original pre-trained model?

I have downloaded a pre-trained EfficientDet D2 model (Tensorflow 2.0) and trained it on some data (about 20000 images with 20 classes). I set the number of steps to 25000 and batch size to 3 (...
Araw's user avatar
  • 103
0 votes
0 answers
59 views

Model not learning anything, what can be the problem?

I've trained a model for heart sound classification with transfer learning (MobileNet) on Physionet dataset, and it works fine. However, when I train it on my own dataset, it seems that it can not ...
Sepehr Golestanian's user avatar
2 votes
1 answer
397 views

How could Bayesian neural networks be used for transfer learning?

In transfer learning, we use big data from similar tasks to learn the parameters of a neural network, and then fine-tune the neural network on our own task that has little data available for it. Here, ...
samsambakster's user avatar
0 votes
1 answer
97 views

Why would the "improvement" be the result of random initialization, and so why should we use multiple runs?

I got this feedback for my thesis paper. The improvement shown in the results section could be the result of random initialization. There should be multiple runs with means and standard deviations. ...
Md. Asif Iqbal Fahim's user avatar
1 vote
0 answers
132 views

How to deal with images that do not contain any object of interest?

I'm currently working on an iOS App where I want to detect if there is a table, chair or bench in the current camera input. My idea was to take the MobileNetV2 model and get it to classify these three ...
Robin's user avatar
  • 11
1 vote
1 answer
99 views

Would this count as a Transfer Learning approach?

I have two datasets, Dataset 1(D1) and Dataset 2(D2). D1 has around 22000 samples, and D2 has around 8000 samples. What I am doing is that I train a Deep Neural Network model with around three layers ...
Ravish Jha's user avatar
0 votes
1 answer
634 views

Validation accuracy very low with transfer learning

I am using MobileNetV3 from TF keras for doing transfer learning; I removed the last layer, added two dense layers, and trained for 20 epochs. How many dense layers should I add after the MobileNet ...
Hossam Alzomor's user avatar
2 votes
3 answers
4k views

What is the difference between feature extraction and fine-tuning in transfer learning?

I'm building a model for facial expression recognition, and I want to use transfer learning. From what I understand, there are different steps to do it. The first is the feature extraction and the ...
Speedskillsx's user avatar
0 votes
1 answer
122 views

How to perform multi-class text classification with a dataset of 80 documents?

I have a training dataset of 80 text documents with an average number of characters in each document of 25000 and 210 unique tags. How can I perform multi-class text classification with such a small ...
Utkarsh Malkoti's user avatar
-1 votes
1 answer
142 views

How to train my model using transfer learning on inception_v3 pre-trained model?

I am trying to train my model to classify 10 classes of hand gestures but I don't get why am I getting validation accuracy approx. double than training accuracy. My dataset is from kaggle: https://www....
Shubham Agrawal's user avatar
6 votes
2 answers
1k views

What does "semantic gap" mean?

I was reading DT-LET: Deep transfer learning by exploring where to transfer, and it contains the following: It should be noted direct use of labeled source domain data on a new scene of target domain ...
Kais Hasan's user avatar
4 votes
1 answer
194 views

What is the relation between self-taught learning and transfer learning?

I am new to transfer learning and I start by reading A Survey on Transfer Learning, and it stated the following: according to different situations of labeled and unlabeled data in the source domain, ...
Kais Hasan's user avatar
1 vote
0 answers
54 views

What is the definition of pre-training?

I want to pre-train a model (combined by two popular modules A and B, and both are large blocks), then fine-tune it on downstream tasks. What if for the weight initialization for pre-training, module ...
zuujhyt's user avatar
  • 11
4 votes
1 answer
4k views

How is few-shot learning different from transfer learning?

To my understanding, transfer learning helps to incorporate data from other related datasets and achieve the task with less labelled data (maybe in 100s of images per category). Few-shot learning ...
Geneveve08's user avatar
3 votes
0 answers
123 views

Why shouldn't batch normalisation layers be learnable during fine-tuning?

I have been reading this TensorFlow tutorial on transfer learning, where they unfroze the whole model and then they say: When you unfreeze a model that contains ...
dato nefaridze's user avatar
2 votes
0 answers
36 views

Literature on the advantages of using an auto-encoder for classification

Given a supervised problem with X, y input pairs, one can do two things for obtaining the function f that maps X with y with Neural Networks (and in general in machine learning): Deploy directly a ...
Tommaso Bendinelli's user avatar
2 votes
1 answer
137 views

Is it possible to use self-supervised learning on different images for the pretext and downstream tasks?

I have just come across the idea of self-supervised learning. It seems that it is possible to get higher accuracies on downstream tasks when the network is trained on pretext tasks. Suppose that I ...
calveeen's user avatar
  • 1,261
2 votes
1 answer
304 views

Is it possible to pre-train a CNN in a self-supervised way so that it can later be used to solve an instance segmentation task?

I would like to use self-supervised learning (SSL) to learn features from images (the dataset consists of similar images with small differences), then use the resulting trained model to bootstrap an ...
Timco Vanco's user avatar
0 votes
1 answer
462 views

Transfer Learning of Numerical Data

It seems like transfer learning is only applicable to neural networks. Is this a correct assumption? While I was looking for examples of Transfer Learning, most seemed to be based on image data, audio ...
Sung Kyu Park's user avatar
2 votes
0 answers
65 views

Is it ok to perform transfer learning with a base model for face recognition to perform one-shot learning for object classification?

I am trying to create a model that is using a one-shot learning approach for a classification task. We do this because we do not have a lot of data and it also seems like a good way to learn this ...
Maciek Woźniak's user avatar
4 votes
1 answer
2k views

Why aren't the BERT layers frozen during fine-tuning tasks?

During transfer learning in computer vision, I've seen that the layers of the base model are frozen if the images aren't too different from the model on which the base model is trained on. However, on ...
Bunny Rabbit's user avatar
3 votes
2 answers
497 views

Can we apply transfer learning between any two different CNN architectures?

There are many types of CNN architectures: LeNet, AlexNet, VGG, GoogLeNet, ResNet, etc. Can we apply transfer learning between any two different CNN architectures? For instance, can we apply transfer ...
The Pointer's user avatar
0 votes
1 answer
65 views

BERT: After pretraining 880000 step, why fine-tune not work? [closed]

I am using pretraining code from https://github.com/NVIDIA/DeepLearningExamples Pretrain parameters: ...
惊天补扣's user avatar
1 vote
0 answers
25 views

Why is domain adaptation and generative modelling for knowledge graphs still not applied widely in enterprise data? What are the challenges?

I see that domain adaptation and transfer learning has been widely adopted in image classification and semantic segmentation analysis. But it's still lacking in providing solutions to enterprise data, ...
Jey 's user avatar
  • 11
3 votes
1 answer
563 views

What's the difference between domain randomization and domain adaptation?

In my understanding, domain randomization is one method of diversifying the dataset to achieve a better shot at domain adaptation. Am I wrong?
Taro Yehai's user avatar
4 votes
2 answers
3k views

What is layer freezing in transfer learning?

Transfer learning consists of taking features learned on one problem and leveraging them on a new, similar problem. In the Transfer Learning, we take layers from a previously trained model and freeze ...
Pluviophile's user avatar
  • 1,273
1 vote
3 answers
216 views

How does batch normalisation actually work?

I actually went through the Keras' batch normalization tutorial and the description there puzzled me more. Here are some facts about batch normalization that I read recently and want a deep ...
Devansh Khandekar's user avatar
0 votes
1 answer
107 views

How to "forward" updated NN model to a transferred model?

I've trained a robot to walk in a straight line for as long as it can (using TD3), and now I'm using that pre-trained model for two new models with separate purposes: 1. Walk to a specific point and ...
pinkie pAI's user avatar
3 votes
1 answer
338 views

Does self-supervised learning require auxiliary tasks?

Self-supervised learning algorithms provide labels automatically. But, it is not clear what else is required for an algorithm to fall under the category "self-supervised": Some say, self-...
Make42's user avatar
  • 163
12 votes
1 answer
5k views

What is the difference between one-shot learning, transfer learning and fine tuning?

Lately, there are lots of posts on one-shot learning. I tried to figure out what it is by reading some articles. To me, it looks like similar to transfer learning, in which we can use pre-trained ...
Hiren Namera's user avatar
1 vote
0 answers
63 views

Is the high dimensionality of input vectors a problem for a radial basis function neural network?

I have a dataset A of videos. I've extracted the feature vector of each video (with a convolutional neural network, via transfer learning) creating a dataset B. Now, every vector of the dataset B has ...
AleWolf's user avatar
  • 167
4 votes
0 answers
50 views

What is the difference between "out-of-distribution (generalisation)" and "(meta)-transfer learning"?

I'm trying to develop a better understanding of the concept of "out-of-distribution" (generalization) in the context of Bengio's "Moving from System 1 DL to System 2 DL" and the concept of "(meta)-...
maxcompression's user avatar
3 votes
1 answer
274 views

How can I detect the frame from video streaming that contains a graffiti on city wall?

I am working on a graffiti detection project. I need to analyze data stream from a camera mounted sideways on a vehicle to identify graffiti on city walls and notify authorities with the single best ...
sgX's user avatar
  • 133