Questions tagged [transfer-learning]

For questions related to transfer learning, a machine learning method that focuses on storing knowledge gained while solving one problem in order to apply this knowledge to a different but related problem.

Filter by
Sorted by
Tagged with
1
vote
0answers
27 views

What is the definition of pre-training?

I want to pre-train a model (combined by two popular modules A and B, and both are large blocks), then fine-tune it on downstream tasks. What if for the weight initialization for pre-training, module ...
1
vote
0answers
34 views

How is few-shot learning different from transfer learning?

To my understanding, transfer learning helps to incorporate data from other related datasets and achieve the task with less labelled data (maybe in 100s of images per category). Few-shot learning ...
0
votes
0answers
18 views

Multi-objective training vs Transfer learning - pros and cons

I'm solving a sequential prediction task that has multiple features attached to each timestamp, and my goal is to predict one of them. However, the feature's label is highly imbalanced: only about 4% ...
3
votes
0answers
31 views

Why shouldn't batch normalisation layers be learnable during fine-tuning?

I have been reading this TensorFlow tutorial on transfer learning, where they unfroze the whole model and then they say: When you unfreeze a model that contains ...
2
votes
0answers
24 views

Literature on the advantages of using an auto-encoder for classification

Given a supervised problem with X, y input pairs, one can do two things for obtaining the function f that maps X with y with Neural Networks (and in general in machine learning): Deploy directly a ...
1
vote
0answers
30 views

Is it possible to use self-supervised learning on different images for the pretext and downstream tasks?

I have just come across the idea of self-supervised learning. It seems that it is possible to get higher accuracies on downstream tasks when the network is trained on pretext tasks. Suppose that I ...
2
votes
1answer
49 views

Is it possible to pre-train a CNN in a self-supervised way so that it can later be used to solve an instance segmentation task?

I would like to use self-supervised learning (SSL) to learn features from images (the dataset consists of similar images with small differences), then use the resulting trained model to bootstrap an ...
-1
votes
1answer
57 views

Transfer Learning of Numerical Data

It seems like transfer learning is only applicable to neural networks. Is this a correct assumption? While I was looking for examples of Transfer Learning, most seemed to be based on image data, audio ...
2
votes
0answers
30 views

Is it ok to perform transfer learning with a base model for face recognition to perform one-shot learning for object classification?

I am trying to create a model that is using a one-shot learning approach for a classification task. We do this because we do not have a lot of data and it also seems like a good way to learn this ...
1
vote
0answers
120 views

Why aren't the BERT layers frozen during fine-tuning tasks?

During transfer learning in computer vision, I've seen that the layers of the base model are frozen if the images aren't too different from the model on which the base model is trained on. However, on ...
2
votes
1answer
65 views

Can we apply transfer learning between any two different CNN architectures?

There are many types of CNN architectures: LeNet, AlexNet, VGG, GoogLeNet, ResNet, etc. Can we apply transfer learning between any two different CNN architectures? For instance, can we apply transfer ...
0
votes
1answer
40 views

BERT: After pretraining 880000 step, why fine-tune not work? [closed]

I am using pretraining code from https://github.com/NVIDIA/DeepLearningExamples Pretrain parameters: ...
1
vote
0answers
15 views

Why is domain adaptation and generative modelling for knowledge graphs still not applied widely in enterprise data? What are the challenges?

I see that domain adaptation and transfer learning has been widely adopted in image classification and semantic segmentation analysis. But it's still lacking in providing solutions to enterprise data, ...
2
votes
0answers
58 views

What's the difference between domain randomization and domain adaptation?

In my understanding, domain randomization is one method of diversifying the dataset to achieve a better shot at domain adaptation. Am I wrong?
0
votes
0answers
16 views

Transfer Learning: Finetune a model with a splitted dataset?

Lets say I want to fine-tune a model. I have a pretrained ResNet model and on top of this model I add some extra layers. And lets say I have a dataset of 10,000 images. The recommended way would be: ...
3
votes
2answers
170 views

What is layer freezing in transfer learning?

Transfer learning consists of taking features learned on one problem and leveraging them on a new, similar problem. In the Transfer Learning, we take layers from a previously trained model and freeze ...
1
vote
0answers
47 views

Reference for Transfer Learning via Final Layers of a Neural Network

Problem (Sketch): I'm interested in a particular formulation of the transfer-learning problem, which, given a trained network $f$ seeks to learn a new network $g$ whose last few layers behave very ...
0
votes
0answers
52 views

Can I train a neural network with multiple datasets (e.g. 25)?

I want to create a neural network that I can train with many datasets (e.g. 20 - 25 datasets). Can I use transfer learning for this? Or is there a better approach than this?
1
vote
3answers
95 views

How does batch normalisation actually work?

I actually went through the Keras' batch normalization tutorial and the description there puzzled me more. Here are some facts about batch normalization that I read recently and want a deep ...
0
votes
1answer
98 views

How to “forward” updated NN model to a transferred model?

I've trained a robot to walk in a straight line for as long as it can (using TD3), and now I'm using that pre-trained model for two new models with separate purposes: 1. Walk to a specific point and ...
1
vote
1answer
105 views

Does self-supervised learning require auxiliary tasks?

Self-supervised learning algorithms provide labels automatically. But, it is not clear what else is required for an algorithm to fall under the category "self-supervised": Some say, self-...
4
votes
1answer
841 views

What is the difference between one-shot learning, transfer learning and fine tuning?

Lately, there are lots of posts on one-shot learning. I tried to figure out what it is by reading some articles. To me, it looks like similar to transfer learning, in which we can use pre-trained ...
1
vote
0answers
27 views

Is the high dimensionality of input vectors a problem for a radial basis function neural network?

I have a dataset A of videos. I've extracted the feature vector of each video (with a convolutional neural network, via transfer learning) creating a dataset B. Now, every vector of the dataset B has ...
0
votes
0answers
32 views

Can neural networks always be assembled like Lego blocks?

BACKGROUND Consider a supervised problem which is based on two scalar features (1) and (2) as well as a third, "time-dependent", feature consisting of a sequence of five values (3)-(7). For ...
1
vote
0answers
192 views

What are the actual math or computer science concepts behind these unfamiliar hyperparameters in the Deep Dream Generator's Deep Style?

I've been playing around with neural style transfer for a about a year now, and I've been doing it with two general approaches. The first has been using a script that is available on the Keras GitHub, ...
3
votes
0answers
33 views

What is the difference between “out-of-distribution (generalisation)” and “(meta)-transfer learning”?

I'm trying to develop a better understanding of the concept of "out-of-distribution" (generalization) in the context of Bengio's "Moving from System 1 DL to System 2 DL" and the concept of "(meta)-...
3
votes
1answer
122 views

How can I detect the frame from video streaming that contains a graffiti on city wall?

I am working on a graffiti detection project. I need to analyze data stream from a camera mounted sideways on a vehicle to identify graffiti on city walls and notify authorities with the single best ...
2
votes
0answers
69 views

Pre-trained Models for Topic Modelling Transfer Learning (LDA)

I've been searching online - and so far, I've been unable to find any publicly-accessible pre-trained models that can be used for LDA Topic Modeling - Transfer Learning. Can anyone share any resources ...
4
votes
3answers
3k views

What are the differences between transfer learning and meta learning?

What are the differences between meta-learning and transfer learning? I have read 2 articles on Quora and TowardDataScience. Meta learning is a part of machine learning theory in which some ...
2
votes
0answers
98 views

What is the difference between using a backbone architecture and transfer learning?

I'm super new to deep learning and computer vision, so this question may sound dumb. In this link (https://github.com/GeorgeSeif/Semantic-Segmentation-Suite), there are pre-trained models (e.g., ...
1
vote
0answers
487 views

Reasoning behind $Zero$ validation accuracy in the following ResNet50 model for classification

I have written this code to classify Cats and dogs using Resnet50. Actually while studying I came to the conclusion that Transfer learning gives very good accuracy for deep learning models, but I ...
1
vote
0answers
17 views

learning object recognition of primitive shapes through transfer learning problem

Question on transfer learning object classification (MobileNet_v2 with 75% number of parameters) with my own synthetic data: I made my own dataset of three shapes: triangles, rectangles and spheres. ...
4
votes
0answers
37 views

How can I improve the performance of a model trained to detect vehicle poses?

I'm looking for some suggestions on how to improve our vehicle image recognition. We have an online marketplace where customers submit photos of their vehicles. The photos need to meet certain ...
3
votes
1answer
65 views

Precise description of one-shot learning

I am working on classifying the Omniglot dataset, and the different papers dealing with this topic describe the problem as one-shot learning (classification). I would like to nail down a precise ...
5
votes
1answer
176 views

What are the real-life applications of transfer learning?

What are the real-life applications of transfer learning in machine learning? I am particularly interested in industrial applications of the concept.
2
votes
0answers
986 views

How to transfer learn Darknet YOLOv3

I've started getting into object detection in image. I have YOLOv3 neural network with Darknet framework. The network is pre-trained from COCO data set. Now I need to do some transfer learning in ...
3
votes
1answer
141 views

Transfer learning to train only for a new class while not affecting the predictions of the other class

I am basically interested in vehicle on the road. YoloV3 pytorch is giving a decent result. So my interested Vehicles Car ...
3
votes
1answer
307 views

How is transfer learning used to mitigate catastrophic forgetting in neural networks?

How can transfer learning be used to mitigate catastrophic forgetting. Could someone elaborate on this?
2
votes
1answer
75 views

What are the most common methods to enable neural networks to adapt to changing environments?

For real applications, concept drifts often exist, i.e., the relationship between the input and output changes overtime. Thus, we need our AI or machine learning system to quickly adapt to the ...
5
votes
2answers
1k views

What is the difference between learning without forgetting and transfer learning?

I would like to incrementally train my model with my current dataset and I asked this question on Github, which is what I'm using SSD MobileNet v1. Someone there told me about learning without ...
1
vote
0answers
43 views

Is convergence to a local minima more likely with transfer learning?

While doing transfer learning where my two problems are face-generation and car-generation is it likely that, if I use the weights of one problem as the initialization of the weights for the other ...
2
votes
0answers
124 views

Paper & code for “unsupervised domain adaptation” for regression task

Does anyone know a paper or code that does "unsupervised domain adaptation" for regression task? I saw most of the papers were benchmarked on classification tasks, not regression. I want to do ...
1
vote
0answers
16 views

Binary annotations on large, heterogenous images

I'm working on a deep learning project and have encountered a problem. The images that I'm using are very large and extremely detailed. They also contain a huge amount of necessary visual information, ...
3
votes
1answer
91 views

Are there any better visual models for transfer rather than ImageNet?

Similar to the recent pushes in Pretrained Language Models (BERT, GPT2, XLNet) I was wondering if such a thrust exists in Computer Vision? From my understanding, it seems the community has converged ...
3
votes
0answers
29 views

Training a reinforcement learning model with multiple images

I am tentatively trying to train a deep reinforcement learning model the maze escaping task, and each time it takes one image as the input (e.g., a different "maze"). Suppose I have about $10K$ ...
5
votes
2answers
288 views

How can I train a neural network for image classification when the dataset is small?

I need to train a convolutional neural network to classify snake images. The problem is that I have only a small number of images available for some snake types. So, what is the best approach to train ...
1
vote
1answer
107 views

Why is my fine-tuned YOLO model detecting other objects as a human?

I am new to deep learning and computer vision. I have a problem where I use the YOLO to detect objects. For my problem, I just want to recognize 1 human only. So, I changed the final YOLO's layer (...
3
votes
1answer
172 views

When doing transfer learning, which initial layers do we need to freeze, and how should I change the last layer for my task?

I want to train a neural network for the detection of a single class, but I will be extending it to detect more classes. To solve this task, I selected the PyTorch framework. I came across transfer ...
2
votes
4answers
196 views

Why do we have to train a model from scratch every time? [closed]

I have started on Andrew Ng's machine learning course. It seems that machine learning is learning correlations with known data based on as many parameters as possible. For example, if we collect data ...
26
votes
4answers
11k views

Is it possible to train a neural network as new classes are given?

I would like to train a neural network (NN) where the output classes are not (all) defined from the start. More and more classes will be introduced later based on incoming data. This means that, every ...