Questions tagged [fine-tuning]

For questions related to the concept of fine-tuning a model (e.g. neural network), which is very related to and sometimes used as a synonym for transfer learning.

Filter by
Sorted by
Tagged with
0
votes
0answers
26 views

How to fine-tune GPT-J with small dataset

I have followed this guide as closely as possible: https://github.com/kingoflolz/mesh-transformer-jax I'm trying to fine-tune GPT-J with a small dataset of ~500 lines: ...
0
votes
0answers
47 views

How to fine-tune a model which was pre-trained on a corpus that contains words with different meanings than the meanings of those words on my corpus?

I have a scenario in which we should leverage previously asked questions (not questions pairs, single question in a column) to locate similar questions within those questions. How can I fine-tune my ...
0
votes
0answers
28 views

How big should the dataset for retraining ssd_mobilenet_v2 be?

I have retrained ssd_mobilenet_v2 using my own dataset with 2 classes (pen or pencil), using object detection API. For my project, I expect users to select specific pencils from all pencils and ...
0
votes
1answer
89 views

Is it possible that the fine-tuned pre-trained model performs worse than the original pre-trained model?

I have downloaded a pre-trained EfficientDet D2 model (Tensorflow 2.0) and trained it on some data (about 20000 images with 20 classes). I set the number of steps to 25000 and batch size to 3 (...
0
votes
1answer
44 views

Does BERT freeze the entire model body when it does fine-tuning?

Recently, I came across the BERT model. I did some research and tried some implementations. I wanted to tackle a NER task, so I chose the BertForSequenceClassifications provided by HuggingFace. ...
1
vote
1answer
59 views

Would this count as a Transfer Learning approach?

I have two datasets, Dataset 1(D1) and Dataset 2(D2). D1 has around 22000 samples, and D2 has around 8000 samples. What I am doing is that I train a Deep Neural Network model with around three layers ...
0
votes
1answer
222 views

What is the difference between feature extraction and fine-tuning in transfer learning?

I'm building a model for facial expression recognition, and I want to use transfer learning. From what I understand, there are different steps to do it. The first is the feature extraction and the ...
0
votes
0answers
19 views

Open Domain Chatbot with intervention

As per my knowledge, there are 2 type of Chatbots- Open Domain and Closed Doman Intent Based. I was wondering how are conversational agents created such that can converse as if they are Open Domain, ...
1
vote
1answer
52 views

Can Facebook's LASER be used like BERT?

Can Facebook's LASER be fine-tuned like BERT for Question Answering tasks or Sentiment Analysis? From my understanding, they created an embedding that allows for similar words in different languages ...
2
votes
0answers
23 views

Adding corpus to BERT for QA

I was wondering about SciBERT's QA abilities using SQuAD. I have a scarce textual dataset consisting of less than 100 files where doctors are discussing cancer in dialogues. I want to add it to ...
1
vote
0answers
26 views

How to scrape product data on supplier websites?

I'm currently trying to build a semantic scraper that can extract product information from different company websites of suppliers in the packaging industry (with as little manual customization per ...
1
vote
0answers
38 views

What is the definition of pre-training?

I want to pre-train a model (combined by two popular modules A and B, and both are large blocks), then fine-tune it on downstream tasks. What if for the weight initialization for pre-training, module ...
3
votes
0answers
42 views

Why shouldn't batch normalisation layers be learnable during fine-tuning?

I have been reading this TensorFlow tutorial on transfer learning, where they unfroze the whole model and then they say: When you unfreeze a model that contains ...
0
votes
0answers
28 views

Is it possible to improve the average precision of YOLO trained on Open Images Dataset by fine-tuning it with COCO?

I consider pre-training a YOLOv5 with Google Open Images Object Detection dataset. The dataset includes general domain categories with ~15 M box samples. After the pre-training is done, I will fine-...
2
votes
0answers
563 views

Why aren't the BERT layers frozen during fine-tuning tasks?

During transfer learning in computer vision, I've seen that the layers of the base model are frozen if the images aren't too different from the model on which the base model is trained on. However, on ...
0
votes
1answer
46 views

BERT: After pretraining 880000 step, why fine-tune not work? [closed]

I am using pretraining code from https://github.com/NVIDIA/DeepLearningExamples Pretrain parameters: ...
4
votes
1answer
2k views

How to fine tune BERT for question answering?

I wish to train two domain-specific models: Domain 1: Constitution and related Legal Documents Domain 2: Technical and related documents. For Domain 1, I've access to a text-corpus with texts from ...
4
votes
1answer
2k views

What is the difference between one-shot learning, transfer learning and fine tuning?

Lately, there are lots of posts on one-shot learning. I tried to figure out what it is by reading some articles. To me, it looks like similar to transfer learning, in which we can use pre-trained ...
1
vote
1answer
805 views

GPT-2: (Hardware) requirements for fine-tuning the 774M model

I wonder if there's anyone who has actually succeeded in fine-tuning GPT-2's 774M model without using cloud TPU's. My GeForce RTX 2070 SUPER couldn't handle it in previous attempts. I'm running ...
2
votes
0answers
180 views

How does one continue the pre-training in BERT?

I need some help with continuing pre-training on Bert. I have a very specific vocabulary and lots of specific abbreviations at hand. I want to do an STS task. Let me specify my task: I have domain-...
2
votes
1answer
77 views

Is my fine-tuned model learning anything at all?

I am practicing with Resnet50 fine-tuning for a binary classification task. Here is my code snippet. ...
3
votes
1answer
367 views

When doing transfer learning, which initial layers do we need to freeze, and how should I change the last layer for my task?

I want to train a neural network for the detection of a single class, but I will be extending it to detect more classes. To solve this task, I selected the PyTorch framework. I came across transfer ...