Questions tagged [transfer-learning]

For questions related to transfer learning, a machine learning method that focuses on storing knowledge gained while solving one problem in order to apply this knowledge to a different but related problem.

Filter by
Sorted by
Tagged with
0 votes
0 answers
14 views

Add adapters to machine translation model [closed]

I'm following this guide which explains how to apply adapters to a model for a binary classification task, and I want to adapt it to a machine translation task. Once I get here I get stuck because the ...
user avatar
  • 1
-1 votes
0 answers
35 views

How to further improve(techniques) transfer learning training results of object detection model [closed]

I am using a pre trained pytorch object detection model for transfer learning on custom data I want to train. I am referring to Building your own object detector — PyTorch vs TensorFlow and how to ...
user avatar
0 votes
0 answers
35 views

Transfer learning on YOLOv5 for character and shape detection

The task is to detect rotated alphanumeric characters embedded on colored shapes. We will have an aerial view of the object (from a UAS: Unarmed Aerial System), something of this sort: (One Uppercase ...
user avatar
  • 121
2 votes
0 answers
48 views

How to choose the new layer and objective function for transfer learning on a neural network?

I have a base model $M$ trained on a data say type 1 for task $T$. Now, I want to update $M$ by applying transfer learning for it to work on data type 2 for the same task $T$. I am very new to AI/ML ...
user avatar
  • 121
1 vote
0 answers
21 views

Transferring a Q-learning policy to larger instances

How do I best transfer and fine-tune a Q-learning policy that was trained on small instances to large instances? Some more details on the problem: I am currently trying to derive a decision policy for ...
user avatar
0 votes
0 answers
40 views

Without using data augmentation gives results better than using data augmentation

I am a beginner to deep learning, I'm doing the image classification problem on a small self plant disease imaging dataset (400 images). I am doing transfer learning (pre-trained ...
user avatar
  • 1
1 vote
0 answers
10 views

Continue teaching pre-trained network without forgetting previous data set

I have a rather interesting problem here; I work in the field of image classification for quality assurance. For this I have a dataset of about 1 million images, which I have used to train different ...
user avatar
  • 131
0 votes
0 answers
38 views

How big should the dataset for retraining ssd_mobilenet_v2 be?

I have retrained ssd_mobilenet_v2 using my own dataset with 2 classes (pen or pencil), using object detection API. For my project, I expect users to select specific pencils from all pencils and ...
user avatar
  • 1
0 votes
0 answers
23 views

What is meant by "training a YOLOv5 on custom dataset"?

I am new to AI/CV domain. I see a lot of tutorials on "training a YOLOv5 on custom dataset". As per my knowledge, Basically, YOLO is a giant neural network trained on a particular set of ...
user avatar
  • 121
0 votes
0 answers
46 views

How to calculate computational efficiency of Deep Learning Models?

I am trying to make a comparison between two simple 5 layer neural network models. One of the models has 3 frozen layers as I've implemented transfer learning in that architecture. The other is ...
user avatar
0 votes
1 answer
32 views

Should I label static objects on video dataset?

I'm using nvidia Transfer Learning Toolkit to detect cars in some video frames. I found some dataset (for example https://www.jpjodoin.com/urbantracker/dataset.html and https://www.kaggle.com/...
user avatar
0 votes
1 answer
204 views

Is it possible that the fine-tuned pre-trained model performs worse than the original pre-trained model?

I have downloaded a pre-trained EfficientDet D2 model (Tensorflow 2.0) and trained it on some data (about 20000 images with 20 classes). I set the number of steps to 25000 and batch size to 3 (...
user avatar
  • 103
0 votes
0 answers
32 views

Model not learning anything, what can be the problem?

I've trained a model for heart sound classification with transfer learning (MobileNet) on Physionet dataset, and it works fine. However, when I train it on my own dataset, it seems that it can not ...
user avatar
2 votes
1 answer
143 views

How could Bayesian neural networks be used for transfer learning?

In transfer learning, we use big data from similar tasks to learn the parameters of a neural network, and then fine-tune the neural network on our own task that has little data available for it. Here, ...
user avatar
0 votes
1 answer
39 views

Why would the "improvement" be the result of random initialization, and so why should we use multiple runs?

I got this feedback for my thesis paper. The improvement shown in the results section could be the result of random initialization. There should be multiple runs with means and standard deviations. ...
user avatar
0 votes
0 answers
28 views

How to work deeper with YOLO v4 scaled

I am pretty new in object detection. When I did some classification using TF with pretrained models I changed some last layers and choosed how many layers I want to train. I don't really get it with ...
user avatar
  • 1
0 votes
0 answers
20 views

Naming convention for deep learning layer sequences ("FC7", "Conv-1-3")

I was looking at the deep learning paper A Target-agnostic Attack on Deep Models and saw this figure (figure 3 on paper) demonstrating the performance of a transfer-learning-based adversarial attack ...
user avatar
  • 1
0 votes
0 answers
8 views

Can I use transfer learning with Bert and none text sequential data?

I'm working on a multiclass classification problem, each row on my dataset have 5 time windows with 23 values on each time window, I would like to use transfer learning using the Bert transformer to ...
user avatar
1 vote
0 answers
31 views

How to deal with images that do not contain any object of interest?

I'm currently working on an iOS App where I want to detect if there is a table, chair or bench in the current camera input. My idea was to take the MobileNetV2 model and get it to classify these three ...
user avatar
  • 11
1 vote
1 answer
65 views

Would this count as a Transfer Learning approach?

I have two datasets, Dataset 1(D1) and Dataset 2(D2). D1 has around 22000 samples, and D2 has around 8000 samples. What I am doing is that I train a Deep Neural Network model with around three layers ...
user avatar
0 votes
1 answer
180 views

Validation accuracy very low with transfer learning

I am using MobileNetV3 from TF keras for doing transfer learning; I removed the last layer, added two dense layers, and trained for 20 epochs. How many dense layers should I add after the MobileNet ...
user avatar
1 vote
2 answers
730 views

What is the difference between feature extraction and fine-tuning in transfer learning?

I'm building a model for facial expression recognition, and I want to use transfer learning. From what I understand, there are different steps to do it. The first is the feature extraction and the ...
user avatar
0 votes
0 answers
50 views

Anything similar to BERT but for pixel-wise embedding in images

In NLP there is BERT which can take a sentence and turn it into an embedding (vector representation) which in some ways encompasses the "meaning" or more precisely the context of the ...
user avatar
0 votes
1 answer
68 views

How to perform multi-class text classification with a dataset of 80 documents?

I have a training dataset of 80 text documents with an average number of characters in each document of 25000 and 210 unique tags. How can I perform multi-class text classification with such a small ...
user avatar
-1 votes
1 answer
48 views

How to train my model using transfer learning on inception_v3 pre-trained model?

I am trying to train my model to classify 10 classes of hand gestures but I don't get why am I getting validation accuracy approx. double than training accuracy. My dataset is from kaggle: https://www....
user avatar
6 votes
2 answers
245 views

What does "semantic gap" mean?

I was reading DT-LET: Deep transfer learning by exploring where to transfer, and it contains the following: It should be noted direct use of labeled source domain data on a new scene of target domain ...
user avatar
4 votes
1 answer
109 views

What is the relation between self-taught learning and transfer learning?

I am new to transfer learning and I start by reading A Survey on Transfer Learning, and it stated the following: according to different situations of labeled and unlabeled data in the source domain, ...
user avatar
1 vote
0 answers
40 views

What is the definition of pre-training?

I want to pre-train a model (combined by two popular modules A and B, and both are large blocks), then fine-tune it on downstream tasks. What if for the weight initialization for pre-training, module ...
user avatar
  • 11
2 votes
1 answer
641 views

How is few-shot learning different from transfer learning?

To my understanding, transfer learning helps to incorporate data from other related datasets and achieve the task with less labelled data (maybe in 100s of images per category). Few-shot learning ...
user avatar
3 votes
0 answers
66 views

Why shouldn't batch normalisation layers be learnable during fine-tuning?

I have been reading this TensorFlow tutorial on transfer learning, where they unfroze the whole model and then they say: When you unfreeze a model that contains ...
user avatar
2 votes
0 answers
31 views

Literature on the advantages of using an auto-encoder for classification

Given a supervised problem with X, y input pairs, one can do two things for obtaining the function f that maps X with y with Neural Networks (and in general in machine learning): Deploy directly a ...
user avatar
2 votes
1 answer
77 views

Is it possible to use self-supervised learning on different images for the pretext and downstream tasks?

I have just come across the idea of self-supervised learning. It seems that it is possible to get higher accuracies on downstream tasks when the network is trained on pretext tasks. Suppose that I ...
user avatar
  • 1,201
2 votes
1 answer
169 views

Is it possible to pre-train a CNN in a self-supervised way so that it can later be used to solve an instance segmentation task?

I would like to use self-supervised learning (SSL) to learn features from images (the dataset consists of similar images with small differences), then use the resulting trained model to bootstrap an ...
user avatar
0 votes
1 answer
205 views

Transfer Learning of Numerical Data

It seems like transfer learning is only applicable to neural networks. Is this a correct assumption? While I was looking for examples of Transfer Learning, most seemed to be based on image data, audio ...
user avatar
2 votes
0 answers
37 views

Is it ok to perform transfer learning with a base model for face recognition to perform one-shot learning for object classification?

I am trying to create a model that is using a one-shot learning approach for a classification task. We do this because we do not have a lot of data and it also seems like a good way to learn this ...
user avatar
2 votes
0 answers
756 views

Why aren't the BERT layers frozen during fine-tuning tasks?

During transfer learning in computer vision, I've seen that the layers of the base model are frozen if the images aren't too different from the model on which the base model is trained on. However, on ...
user avatar
3 votes
2 answers
171 views

Can we apply transfer learning between any two different CNN architectures?

There are many types of CNN architectures: LeNet, AlexNet, VGG, GoogLeNet, ResNet, etc. Can we apply transfer learning between any two different CNN architectures? For instance, can we apply transfer ...
user avatar
0 votes
1 answer
49 views

BERT: After pretraining 880000 step, why fine-tune not work? [closed]

I am using pretraining code from https://github.com/NVIDIA/DeepLearningExamples Pretrain parameters: ...
user avatar
  • 101
1 vote
0 answers
19 views

Why is domain adaptation and generative modelling for knowledge graphs still not applied widely in enterprise data? What are the challenges?

I see that domain adaptation and transfer learning has been widely adopted in image classification and semantic segmentation analysis. But it's still lacking in providing solutions to enterprise data, ...
user avatar
  • 11
2 votes
1 answer
208 views

What's the difference between domain randomization and domain adaptation?

In my understanding, domain randomization is one method of diversifying the dataset to achieve a better shot at domain adaptation. Am I wrong?
user avatar
3 votes
2 answers
508 views

What is layer freezing in transfer learning?

Transfer learning consists of taking features learned on one problem and leveraging them on a new, similar problem. In the Transfer Learning, we take layers from a previously trained model and freeze ...
user avatar
  • 1,133
0 votes
0 answers
48 views

Reference for Transfer Learning via Final Layers of a Neural Network

Problem (Sketch): I'm interested in a particular formulation of the transfer-learning problem, which, given a trained network $f$ seeks to learn a new network $g$ whose last few layers behave very ...
user avatar
1 vote
3 answers
171 views

How does batch normalisation actually work?

I actually went through the Keras' batch normalization tutorial and the description there puzzled me more. Here are some facts about batch normalization that I read recently and want a deep ...
user avatar
0 votes
1 answer
104 views

How to "forward" updated NN model to a transferred model?

I've trained a robot to walk in a straight line for as long as it can (using TD3), and now I'm using that pre-trained model for two new models with separate purposes: 1. Walk to a specific point and ...
user avatar
3 votes
1 answer
206 views

Does self-supervised learning require auxiliary tasks?

Self-supervised learning algorithms provide labels automatically. But, it is not clear what else is required for an algorithm to fall under the category "self-supervised": Some say, self-...
user avatar
  • 143
5 votes
1 answer
2k views

What is the difference between one-shot learning, transfer learning and fine tuning?

Lately, there are lots of posts on one-shot learning. I tried to figure out what it is by reading some articles. To me, it looks like similar to transfer learning, in which we can use pre-trained ...
user avatar
1 vote
0 answers
31 views

Is the high dimensionality of input vectors a problem for a radial basis function neural network?

I have a dataset A of videos. I've extracted the feature vector of each video (with a convolutional neural network, via transfer learning) creating a dataset B. Now, every vector of the dataset B has ...
user avatar
  • 157
3 votes
0 answers
40 views

What is the difference between "out-of-distribution (generalisation)" and "(meta)-transfer learning"?

I'm trying to develop a better understanding of the concept of "out-of-distribution" (generalization) in the context of Bengio's "Moving from System 1 DL to System 2 DL" and the concept of "(meta)-...
user avatar
3 votes
1 answer
209 views

How can I detect the frame from video streaming that contains a graffiti on city wall?

I am working on a graffiti detection project. I need to analyze data stream from a camera mounted sideways on a vehicle to identify graffiti on city walls and notify authorities with the single best ...
user avatar
  • 133
7 votes
4 answers
6k views

What are the differences between transfer learning and meta learning?

What are the differences between meta-learning and transfer learning? I have read 2 articles on Quora and TowardDataScience. Meta learning is a part of machine learning theory in which some ...
user avatar
  • 133