Questions tagged [fine-tuning]

For questions related to the concept of fine-tuning a model (e.g. neural network), which is very related to and sometimes used as a synonym for transfer learning.

Filter by
Sorted by
Tagged with
1
vote
1answer
34 views

Would this count as a Transfer Learning approach?

I have two datasets, Dataset 1(D1) and Dataset 2(D2). D1 has around 22000 samples, and D2 has around 8000 samples. What I am doing is that I train a Deep Neural Network model with around three layers ...
0
votes
1answer
21 views

What is the difference between feature extraction and fine-tuning in transfer learning?

I'm building a model for facial expression recognition, and I want to use transfer learning. From what I understand, there are different steps to do it. The first is the feature extraction and the ...
0
votes
0answers
14 views

Open Domain Chatbot with intervention

As per my knowledge, there are 2 type of Chatbots- Open Domain and Closed Doman Intent Based. I was wondering how are conversational agents created such that can converse as if they are Open Domain, ...
1
vote
1answer
35 views

Can Facebook's LASER be used like BERT?

Can Facebook's LASER be fine-tuned like BERT for Question Answering tasks or Sentiment Analysis? From my understanding, they created an embedding that allows for similar words in different languages ...
2
votes
0answers
20 views

Adding corpus to BERT for QA

I was wondering about SciBERT's QA abilities using SQuAD. I have a scarce textual dataset consisting of less than 100 files where doctors are discussing cancer in dialogues. I want to add it to ...
1
vote
0answers
20 views

How to scrape product data on supplier websites?

I'm currently trying to build a semantic scraper that can extract product information from different company websites of suppliers in the packaging industry (with as little manual customization per ...
1
vote
0answers
30 views

What is the definition of pre-training?

I want to pre-train a model (combined by two popular modules A and B, and both are large blocks), then fine-tune it on downstream tasks. What if for the weight initialization for pre-training, module ...
3
votes
0answers
37 views

Why shouldn't batch normalisation layers be learnable during fine-tuning?

I have been reading this TensorFlow tutorial on transfer learning, where they unfroze the whole model and then they say: When you unfreeze a model that contains ...
0
votes
0answers
26 views

Is it possible to improve the average precision of YOLO trained on Open Images Dataset by fine-tuning it with COCO?

I consider pre-training a YOLOv5 with Google Open Images Object Detection dataset. The dataset includes general domain categories with ~15 M box samples. After the pre-training is done, I will fine-...
2
votes
0answers
348 views

Why aren't the BERT layers frozen during fine-tuning tasks?

During transfer learning in computer vision, I've seen that the layers of the base model are frozen if the images aren't too different from the model on which the base model is trained on. However, on ...
0
votes
1answer
41 views

BERT: After pretraining 880000 step, why fine-tune not work? [closed]

I am using pretraining code from https://github.com/NVIDIA/DeepLearningExamples Pretrain parameters: ...
0
votes
0answers
18 views

Transfer Learning: Finetune a model with a splitted dataset?

Lets say I want to fine-tune a model. I have a pretrained ResNet model and on top of this model I add some extra layers. And lets say I have a dataset of 10,000 images. The recommended way would be: ...
4
votes
1answer
2k views

How to fine tune BERT for question answering?

I wish to train two domain-specific models: Domain 1: Constitution and related Legal Documents Domain 2: Technical and related documents. For Domain 1, I've access to a text-corpus with texts from ...
4
votes
1answer
1k views

What is the difference between one-shot learning, transfer learning and fine tuning?

Lately, there are lots of posts on one-shot learning. I tried to figure out what it is by reading some articles. To me, it looks like similar to transfer learning, in which we can use pre-trained ...
1
vote
1answer
640 views

GPT-2: (Hardware) requirements for fine-tuning the 774M model

I wonder if there's anyone who has actually succeeded in fine-tuning GPT-2's 774M model without using cloud TPU's. My GeForce RTX 2070 SUPER couldn't handle it in previous attempts. I'm running ...
2
votes
1answer
69 views

Is my fine-tuned model learning anything at all?

I am practicing with Resnet50 fine-tuning for a binary classification task. Here is my code snippet. ...
3
votes
1answer
263 views

When doing transfer learning, which initial layers do we need to freeze, and how should I change the last layer for my task?

I want to train a neural network for the detection of a single class, but I will be extending it to detect more classes. To solve this task, I selected the PyTorch framework. I came across transfer ...