Questions tagged [fine-tuning]

For questions related to the concept of fine-tuning a model (e.g. neural network), which is very related to and sometimes used as a synonym for transfer learning.

15 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
3 votes
0 answers
67 views

Why shouldn't batch normalisation layers be learnable during fine-tuning?

I have been reading this TensorFlow tutorial on transfer learning, where they unfroze the whole model and then they say: When you unfreeze a model that contains ...
user avatar
2 votes
0 answers
24 views

Adding corpus to BERT for QA

I was wondering about SciBERT's QA abilities using SQuAD. I have a scarce textual dataset consisting of less than 100 files where doctors are discussing cancer in dialogues. I want to add it to ...
user avatar
2 votes
0 answers
757 views

Why aren't the BERT layers frozen during fine-tuning tasks?

During transfer learning in computer vision, I've seen that the layers of the base model are frozen if the images aren't too different from the model on which the base model is trained on. However, on ...
user avatar
2 votes
0 answers
250 views

How does one continue the pre-training in BERT?

I need some help with continuing pre-training on Bert. I have a very specific vocabulary and lots of specific abbreviations at hand. I want to do an STS task. Let me specify my task: I have domain-...
user avatar
2 votes
1 answer
84 views

Is my fine-tuned model learning anything at all?

I am practicing with Resnet50 fine-tuning for a binary classification task. Here is my code snippet. ...
user avatar
1 vote
1 answer
85 views

Can Facebook's LASER be used like BERT?

Can Facebook's LASER be fine-tuned like BERT for Question Answering tasks or Sentiment Analysis? From my understanding, they created an embedding that allows for similar words in different languages ...
user avatar
1 vote
0 answers
30 views

How to scrape product data on supplier websites?

I'm currently trying to build a semantic scraper that can extract product information from different company websites of suppliers in the packaging industry (with as little manual customization per ...
user avatar
1 vote
0 answers
40 views

What is the definition of pre-training?

I want to pre-train a model (combined by two popular modules A and B, and both are large blocks), then fine-tune it on downstream tasks. What if for the weight initialization for pre-training, module ...
user avatar
  • 11
0 votes
0 answers
16 views

How to fine-tune RoBERTa using Triplet Objective Function

I want to see if we can improve the triplet objective function of SBERT by slightly tweaking the equation terms. To do so, In your opinion, what's the easiest way to fine-tune RoBERTa? How can I ...
user avatar
  • 103
0 votes
0 answers
14 views

How would you fine tune the model on images of larger size?

Let's say you have trained a model for image classification, on images of a defined size (H x W x Ch). How would you fine tune the model on images of larger size?
user avatar
  • 1
0 votes
1 answer
633 views

How to fine-tune GPT-J with small dataset

I have followed this guide as closely as possible: https://github.com/kingoflolz/mesh-transformer-jax I'm trying to fine-tune GPT-J with a small dataset of ~500 lines: ...
user avatar
0 votes
0 answers
65 views

How to fine-tune a model which was pre-trained on a corpus that contains words with different meanings than the meanings of those words on my corpus?

I have a scenario in which we should leverage previously asked questions (not questions pairs, single question in a column) to locate similar questions within those questions. How can I fine-tune my ...
user avatar
  • 1
0 votes
0 answers
38 views

How big should the dataset for retraining ssd_mobilenet_v2 be?

I have retrained ssd_mobilenet_v2 using my own dataset with 2 classes (pen or pencil), using object detection API. For my project, I expect users to select specific pencils from all pencils and ...
user avatar
  • 1
0 votes
0 answers
21 views

Open Domain Chatbot with intervention

As per my knowledge, there are 2 type of Chatbots- Open Domain and Closed Doman Intent Based. I was wondering how are conversational agents created such that can converse as if they are Open Domain, ...
user avatar
0 votes
0 answers
31 views

Is it possible to improve the average precision of YOLO trained on Open Images Dataset by fine-tuning it with COCO?

I consider pre-training a YOLOv5 with Google Open Images Object Detection dataset. The dataset includes general domain categories with ~15 M box samples. After the pre-training is done, I will fine-...
user avatar
  • 101