Questions tagged [transfer-learning]

For questions related to transfer learning, a machine learning method that focuses on storing knowledge gained while solving one problem in order to apply this knowledge to a different but related problem.

Filter by
Sorted by
Tagged with
-1
votes
0answers
24 views

‘What does ‘truncating final blocks from pretrained models’ mean? Please give concrete examples with code

‘What does ‘truncating final blocks from pretrained models’ mean? Please give concrete examples with code. In the paper, “CheXtransfer: Performance and Parameter Efficiency of ImageNet Models for ...
3
votes
0answers
34 views

How to choose the new layer and objective function for transfer learning on a neural network?

I have a base model $M$ trained on a data say type 1 for task $T$. Now, I want to update $M$ by applying transfer learning for it to work on data type 2 for the same task $T$. I am very new to AI/ML ...
1
vote
0answers
19 views

Transferring a Q-learning policy to larger instances

How do I best transfer and fine-tune a Q-learning policy that was trained on small instances to large instances? Some more details on the problem: I am currently trying to derive a decision policy for ...
0
votes
0answers
21 views

Without using data augmentation gives results better than using data augmentation

I am a beginner to deep learning, I'm doing the image classification problem on a small self plant disease imaging dataset (400 images). I am doing transfer learning (pre-trained ...
1
vote
0answers
10 views

Continue teaching pre-trained network without forgetting previous data set

I have a rather interesting problem here; I work in the field of image classification for quality assurance. For this I have a dataset of about 1 million images, which I have used to train different ...
0
votes
0answers
30 views

How big should the dataset for retraining ssd_mobilenet_v2 be?

I have retrained ssd_mobilenet_v2 using my own dataset with 2 classes (pen or pencil), using object detection API. For my project, I expect users to select specific pencils from all pencils and ...
0
votes
0answers
16 views

What is meant by "training a YOLOv5 on custom dataset"?

I am new to AI/CV domain. I see a lot of tutorials on "training a YOLOv5 on custom dataset". As per my knowledge, Basically, YOLO is a giant neural network trained on a particular set of ...
0
votes
0answers
14 views

How to calculate computational efficiency of Deep Learning Models?

I am trying to make a comparison between two simple 5 layer neural network models. One of the models has 3 frozen layers as I've implemented transfer learning in that architecture. The other is ...
0
votes
1answer
29 views

Should I label static objects on video dataset?

I'm using nvidia Transfer Learning Toolkit to detect cars in some video frames. I found some dataset (for example https://www.jpjodoin.com/urbantracker/dataset.html and https://www.kaggle.com/...
0
votes
1answer
92 views

Is it possible that the fine-tuned pre-trained model performs worse than the original pre-trained model?

I have downloaded a pre-trained EfficientDet D2 model (Tensorflow 2.0) and trained it on some data (about 20000 images with 20 classes). I set the number of steps to 25000 and batch size to 3 (...
0
votes
0answers
29 views

Model not learning anything, what can be the problem?

I've trained a model for heart sound classification with transfer learning (MobileNet) on Physionet dataset, and it works fine. However, when I train it on my own dataset, it seems that it can not ...
2
votes
1answer
65 views

How could Bayesian neural networks be used for transfer learning?

In transfer learning, we use big data from similar tasks to learn the parameters of a neural network, and then fine-tune the neural network on our own task that has little data available for it. Here, ...
0
votes
1answer
38 views

Why would the "improvement" be the result of random initialization, and so why should we use multiple runs?

I got this feedback for my thesis paper. The improvement shown in the results section could be the result of random initialization. There should be multiple runs with means and standard deviations. ...
0
votes
0answers
25 views

How to work deeper with YOLO v4 scaled

I am pretty new in object detection. When I did some classification using TF with pretrained models I changed some last layers and choosed how many layers I want to train. I don't really get it with ...
0
votes
0answers
17 views

Naming convention for deep learning layer sequences ("FC7", "Conv-1-3")

I was looking at the deep learning paper A Target-agnostic Attack on Deep Models and saw this figure (figure 3 on paper) demonstrating the performance of a transfer-learning-based adversarial attack ...
0
votes
0answers
7 views

Can I use transfer learning with Bert and none text sequential data?

I'm working on a multiclass classification problem, each row on my dataset have 5 time windows with 23 values on each time window, I would like to use transfer learning using the Bert transformer to ...
1
vote
0answers
26 views

How to deal with images that do not contain any object of interest?

I'm currently working on an iOS App where I want to detect if there is a table, chair or bench in the current camera input. My idea was to take the MobileNetV2 model and get it to classify these three ...
1
vote
1answer
60 views

Would this count as a Transfer Learning approach?

I have two datasets, Dataset 1(D1) and Dataset 2(D2). D1 has around 22000 samples, and D2 has around 8000 samples. What I am doing is that I train a Deep Neural Network model with around three layers ...
0
votes
1answer
62 views

Validation accuracy very low with transfer learning

I am using MobileNetV3 from TF keras for doing transfer learning; I removed the last layer, added two dense layers, and trained for 20 epochs. How many dense layers should I add after the MobileNet ...
0
votes
1answer
230 views

What is the difference between feature extraction and fine-tuning in transfer learning?

I'm building a model for facial expression recognition, and I want to use transfer learning. From what I understand, there are different steps to do it. The first is the feature extraction and the ...
0
votes
0answers
37 views

Anything similar to BERT but for pixel-wise embedding in images

In NLP there is BERT which can take a sentence and turn it into an embedding (vector representation) which in some ways encompasses the "meaning" or more precisely the context of the ...
0
votes
1answer
44 views

How to perform multi-class text classification with a dataset of 80 documents?

I have a training dataset of 80 text documents with an average number of characters in each document of 25000 and 210 unique tags. How can I perform multi-class text classification with such a small ...
1
vote
0answers
207 views

What could be a good way to visualise the feature extraction process with MobileNet?

I am trying to create a visualisation for how transfer learning (feature extraction in particular) works with MobileNet. With the ml5.js library, you can extract a ...
-1
votes
1answer
39 views

How to train my model using transfer learning on inception_v3 pre-trained model?

I am trying to train my model to classify 10 classes of hand gestures but I don't get why am I getting validation accuracy approx. double than training accuracy. My dataset is from kaggle: https://www....
6
votes
2answers
140 views

What does "semantic gap" mean?

I was reading DT-LET: Deep transfer learning by exploring where to transfer, and it contains the following: It should be noted direct use of labeled source domain data on a new scene of target domain ...
4
votes
1answer
86 views

What is the relation between self-taught learning and transfer learning?

I am new to transfer learning and I start by reading A Survey on Transfer Learning, and it stated the following: according to different situations of labeled and unlabeled data in the source domain, ...
1
vote
0answers
38 views

What is the definition of pre-training?

I want to pre-train a model (combined by two popular modules A and B, and both are large blocks), then fine-tune it on downstream tasks. What if for the weight initialization for pre-training, module ...
2
votes
1answer
125 views

How is few-shot learning different from transfer learning?

To my understanding, transfer learning helps to incorporate data from other related datasets and achieve the task with less labelled data (maybe in 100s of images per category). Few-shot learning ...
0
votes
0answers
20 views

Multi-objective training vs Transfer learning - pros and cons

I'm solving a sequential prediction task that has multiple features attached to each timestamp, and my goal is to predict one of them. However, the feature's label is highly imbalanced: only about 4% ...
3
votes
0answers
44 views

Why shouldn't batch normalisation layers be learnable during fine-tuning?

I have been reading this TensorFlow tutorial on transfer learning, where they unfroze the whole model and then they say: When you unfreeze a model that contains ...
2
votes
0answers
31 views

Literature on the advantages of using an auto-encoder for classification

Given a supervised problem with X, y input pairs, one can do two things for obtaining the function f that maps X with y with Neural Networks (and in general in machine learning): Deploy directly a ...
2
votes
1answer
62 views

Is it possible to use self-supervised learning on different images for the pretext and downstream tasks?

I have just come across the idea of self-supervised learning. It seems that it is possible to get higher accuracies on downstream tasks when the network is trained on pretext tasks. Suppose that I ...
2
votes
1answer
128 views

Is it possible to pre-train a CNN in a self-supervised way so that it can later be used to solve an instance segmentation task?

I would like to use self-supervised learning (SSL) to learn features from images (the dataset consists of similar images with small differences), then use the resulting trained model to bootstrap an ...
0
votes
1answer
141 views

Transfer Learning of Numerical Data

It seems like transfer learning is only applicable to neural networks. Is this a correct assumption? While I was looking for examples of Transfer Learning, most seemed to be based on image data, audio ...
2
votes
0answers
33 views

Is it ok to perform transfer learning with a base model for face recognition to perform one-shot learning for object classification?

I am trying to create a model that is using a one-shot learning approach for a classification task. We do this because we do not have a lot of data and it also seems like a good way to learn this ...
2
votes
0answers
575 views

Why aren't the BERT layers frozen during fine-tuning tasks?

During transfer learning in computer vision, I've seen that the layers of the base model are frozen if the images aren't too different from the model on which the base model is trained on. However, on ...
3
votes
2answers
128 views

Can we apply transfer learning between any two different CNN architectures?

There are many types of CNN architectures: LeNet, AlexNet, VGG, GoogLeNet, ResNet, etc. Can we apply transfer learning between any two different CNN architectures? For instance, can we apply transfer ...
0
votes
1answer
46 views

BERT: After pretraining 880000 step, why fine-tune not work? [closed]

I am using pretraining code from https://github.com/NVIDIA/DeepLearningExamples Pretrain parameters: ...
1
vote
0answers
16 views

Why is domain adaptation and generative modelling for knowledge graphs still not applied widely in enterprise data? What are the challenges?

I see that domain adaptation and transfer learning has been widely adopted in image classification and semantic segmentation analysis. But it's still lacking in providing solutions to enterprise data, ...
2
votes
1answer
140 views

What's the difference between domain randomization and domain adaptation?

In my understanding, domain randomization is one method of diversifying the dataset to achieve a better shot at domain adaptation. Am I wrong?
3
votes
2answers
297 views

What is layer freezing in transfer learning?

Transfer learning consists of taking features learned on one problem and leveraging them on a new, similar problem. In the Transfer Learning, we take layers from a previously trained model and freeze ...
1
vote
0answers
48 views

Reference for Transfer Learning via Final Layers of a Neural Network

Problem (Sketch): I'm interested in a particular formulation of the transfer-learning problem, which, given a trained network $f$ seeks to learn a new network $g$ whose last few layers behave very ...
1
vote
3answers
145 views

How does batch normalisation actually work?

I actually went through the Keras' batch normalization tutorial and the description there puzzled me more. Here are some facts about batch normalization that I read recently and want a deep ...
0
votes
1answer
102 views

How to "forward" updated NN model to a transferred model?

I've trained a robot to walk in a straight line for as long as it can (using TD3), and now I'm using that pre-trained model for two new models with separate purposes: 1. Walk to a specific point and ...
2
votes
1answer
178 views

Does self-supervised learning require auxiliary tasks?

Self-supervised learning algorithms provide labels automatically. But, it is not clear what else is required for an algorithm to fall under the category "self-supervised": Some say, self-...
4
votes
1answer
2k views

What is the difference between one-shot learning, transfer learning and fine tuning?

Lately, there are lots of posts on one-shot learning. I tried to figure out what it is by reading some articles. To me, it looks like similar to transfer learning, in which we can use pre-trained ...
1
vote
0answers
28 views

Is the high dimensionality of input vectors a problem for a radial basis function neural network?

I have a dataset A of videos. I've extracted the feature vector of each video (with a convolutional neural network, via transfer learning) creating a dataset B. Now, every vector of the dataset B has ...
1
vote
0answers
208 views

What are the actual math or computer science concepts behind these unfamiliar hyperparameters in the Deep Dream Generator's Deep Style?

I've been playing around with neural style transfer for a about a year now, and I've been doing it with two general approaches. The first has been using a script that is available on the Keras GitHub, ...
3
votes
0answers
39 views

What is the difference between "out-of-distribution (generalisation)" and "(meta)-transfer learning"?

I'm trying to develop a better understanding of the concept of "out-of-distribution" (generalization) in the context of Bengio's "Moving from System 1 DL to System 2 DL" and the concept of "(meta)-...
3
votes
1answer
175 views

How can I detect the frame from video streaming that contains a graffiti on city wall?

I am working on a graffiti detection project. I need to analyze data stream from a camera mounted sideways on a vehicle to identify graffiti on city walls and notify authorities with the single best ...