Questions tagged [fine-tuning]
For questions related to the concept of fine-tuning a model (e.g. neural network), which is very related to and sometimes used as a synonym for transfer learning.
53
questions
0
votes
0
answers
29
views
How can I teach a book to an LLM?
I am trying to find out how I can teach the content of a whole, multiple hundert pages book to an LLM so that it "knows" all details and can be queried, give summaries etc. The book is one ...
- 1
0
votes
0
answers
20
views
Any research in "probe-tuning" of LLMs?
Is there any research in "probe-tuning" of LLMs, i.e., tuning LLM's parameter weights such that a specific probe (classifier) is more reliably detecting certain markers throughout the ...
- 101
0
votes
0
answers
15
views
can I finetune a model on Azure for information extraction based on "question", "context", and "answer" training data?
I am working on extracting certain fields from a large corpus.
I was looking at finetuning an LLM on Azure for the task. I think finetuning is the right idea (as opposed to vector databases, or RAG), ...
0
votes
0
answers
168
views
Optimal Quantity of Training Data for Fine-Tuning an LLM: Is Bigger Always Better?
I am currently working on fine-tuning an LLM for a specific task, and I am trying to determine the optimal size for my training dataset. Intuitively, one might think that the more data, the better. ...
- 544
1
vote
2
answers
50
views
What is the difference betwen fine runing and rlhf for llm?
I am confused about the difference betwen fine runing and rlhf for llm. When to use what? I know RLHF need to creating a reward model which at furst rates responses to align the responses to the ...
- 253
0
votes
0
answers
13
views
is it possible to make use of classes of coco-pretrained weights on custom dataset just training on custom dataset?
I am trying to use detcetron2 panoptic_FPN and panoptic_deeplab models for optimization on custom dataset.
You might already now that coco has around 133 classes (both thing and stuff). And my custom ...
0
votes
2
answers
61
views
Does fine-tuning a multilingual transformer model allow it to generalize to languages unseen in the fine-tuning dataset?
Example: https://huggingface.co/google/umt5-base
Note: UMT5 was only pre-trained on mC4 excluding any supervised training. Therefore, this model has to be fine-tuned before it is useable on a ...
- 103
0
votes
0
answers
85
views
How to fine-tune a pre-trained model for customer service like tasks chatbot?
I want to make a trainable conversational bot that can respond to customer service questions. The bot should be able to adapt to different domains based on the dataset provided by the user (a company ...
0
votes
0
answers
16
views
How to extract body of a base-model and fine tune with that body on different shape dataset like this situation
In BERT like transformer model (I am not talking about BERT in this thread), it has 2 training objectives Masked Language Modeling and Next sentence prediction right? and BERT model is also supports ...
- 158
0
votes
1
answer
123
views
Fine-tuning or Prompt Engineering or both?
We have a dataset of legal cases, academic papers etc which we will load into a vector database. We want to develop an agent that will allow a user to enter a specific legal issue, the agent then will ...
- 1
2
votes
1
answer
787
views
Should I be layer freezing when fine-tuning an LLM?
I've had it in my head that generally speaking, it's better to freeze layers when fine-tuning an LLM, as per this quote from HuggingFace's article:
PEFT approaches only fine-tune a small number of (...
0
votes
0
answers
287
views
fine-tune nanoGPT for instructions
I've been playing around with nanoGPT, and recently I decided I wanted to fine-tune it using the dolly instruction set. This data set consists of roughly 15k examples and each example has the ...
- 101
1
vote
1
answer
212
views
What researched-backed findings is there for prompting LLM’s / GPT-4 to give specific information or actionable plans?
I have learned a bit recently about prompt strategies. For example, there was a paper about how just by saying “Let’s think step by step” can increase answer quality by like 40%. I have also come to ...
- 123
1
vote
1
answer
109
views
Teaching an LLM about daily updated machine-readable information so it can respond questions
I’m quite new in this field, and despite having spent some good amount of time learning the ins and outs of frameworks like LangChain, and browsing around the internet quite a bit, I still don’t know ...
- 121
1
vote
2
answers
124
views
4 Questions on Transformers [closed]
Assume the transformer is trained on 512 max length sentences:
can we fine-tune it on 256 max-length sentences?
If we can fine-tune it, how is it even possible because the input shapes are different,...
- 158
0
votes
1
answer
76
views
Fine-Tuning T5 with specific penalty
Currently I am finetuning transformers T5 model for translation task. As part of the dataset, I am given sentences in Japanese, their translation to English, and ...
- 121
1
vote
1
answer
370
views
Creating a support chat bot for my business
I am trying to create a kind of Support Bot to answer my clients about specific technical details about WordPress plugins that I sell.
The goal is that the /completitions api would be feeded a prompt ...
0
votes
0
answers
267
views
When would you use prompting vs. fine tuning?
I would like to hear your thoughts on when is it appropriate to use prompting vs. fine-tuning.
Does one make more sense for specific tasks than the other.
Kindly elaborate.
1
vote
2
answers
349
views
Does layer freezing offer other benefits other than to reduce computational time in gradient descent?
In Deep Learning and Transfer Learning, does layer freezing offer other benefits other than to reduce computational time in gradient descent?
Assuming I train a neural network on task A to derive ...
- 21
1
vote
1
answer
144
views
What background should I have before starting to fine tune a Large Language Model?
I want to know what things I should be learning before trying to fine-tune or for that matter working with a large language model.
In my case, I am trying to fine-tune bloom (https://huggingface.co/...
- 23
9
votes
2
answers
3k
views
Are GPT-3.5 series models based on GPT-3?
In the official blog post about ChatGPT from OpenAI, there is this paragraph explaining how ChatGPT model was trained:
We trained this model using Reinforcement Learning from Human Feedback
(RLHF), ...
- 193
2
votes
1
answer
115
views
Would a transformer trained on highly specific material be as usable as a commercial product like ChatGPT?
Soft question here.
I was recently learning a bit about how it is feasible to train a transformer on a personal computer like an M1 Mac. I have been told that the model could have 1-3 million ...
- 123
0
votes
0
answers
42
views
What is a good strategy for breaking up content into prompts and completions for OpenAI fine tuning?
I want to train a fine-tuned openai model to know more about specific Judo throws and training methodologies. I have a bunch of documents I have written on Judo throws that I would like to use for ...
- 101
1
vote
1
answer
476
views
For specific tasks, is it better to fine-tune models on examples or just use prompting with the context of the task?
These days large language models cover a vast amount of topics and information, but I wanted to understand: For specific tasks, is it better to fine-tune models on examples or just use prompting with ...
- 113
0
votes
0
answers
41
views
Transfer learning (or fine-tuning) a pre-trained model on multiple features?
I am currently fine-tuning a sentiment analysis bert-based model using PyTorch Trainer from hugging face. So far, so good.
I have easily managed to fine-tune the model on my text data. However, I'd ...
- 101
1
vote
1
answer
3k
views
Fine-tune GPT-Neo with prompt and completion?
I'm new to AI and machine learning.
To fine-tune GPT-3, I understand that we need a set of training examples that each consist of a single input ("prompt") and its associated output ("...
- 111
2
votes
0
answers
739
views
What is the difference between prompt tuning and prefix tuning?
I read prompt tuning and prefix tuning are two effective mechanisms to leverage frozen language models to perform downstream tasks. What is the difference between the two and how they work really?
...
- 253
0
votes
0
answers
243
views
Combining fine-tuning BERT and cross validation for hyperparameter selections
Is it possible to combine cross-validation procedure and hyper-parameter tuning for fine-tuning bert for a classification task?
The idea is the following:
Choose a set of set of hyperparameters {H,H1,...
- 113
1
vote
0
answers
22
views
How can (pretrained) language models actively seek additional training data - possibly reference request?
I am reading the paper "Large Language Models Can Self-Improve" https://arxiv.org/abs/2210.11610 in which the authors consider that LLM can generate Chain-of-Thoughts sequences and even ...
- 823
1
vote
1
answer
36
views
Fine tuning a Deep Learning model post training
I have trained a CNN in a binary classification problem, however the original problem has 6 different classes, of which, I am only interested in classifying one, so if it is that certain class or not....
- 121
0
votes
1
answer
42
views
can I add to a language model a prompt with output example?
I want to finetune GPT2 to extract relevant data from a given text. So for (a trivial) example, given the text "the car was manufactured in X, can reach Y km/h, and has Z horse powers", my ...
- 371
0
votes
2
answers
834
views
How to combine pretrained language models with additional feature set?
Are there any techniques to combine a feature set (other than the text itself) with pretrained language models.
Let's say I have a random NLP task that tries to predict a binary class label based on e....
- 101
4
votes
1
answer
3k
views
What is the difference between fine tuning and variants of few shot learning? [duplicate]
I am trying to understand the concept of fine-tuning and few-shot learning.
I understand the need for fine-tuning. It is essentially tuning a pre-trained model to a specific downstream task.
However, ...
- 253
0
votes
2
answers
5k
views
How to fine-tune GPT-J with small dataset
I have followed this guide as closely as possible: https://github.com/kingoflolz/mesh-transformer-jax
I'm trying to fine-tune GPT-J with a small dataset of ~500 lines:
...
- 101
1
vote
0
answers
113
views
How to fine-tune a model which was pre-trained on a corpus that contains words with different meanings than the meanings of those words on my corpus?
I have a scenario in which we should leverage previously asked questions (not questions pairs, single question in a column) to locate similar questions within those questions.
How can I fine-tune my ...
- 11
0
votes
1
answer
947
views
Is it possible that the fine-tuned pre-trained model performs worse than the original pre-trained model?
I have downloaded a pre-trained EfficientDet D2 model (Tensorflow 2.0) and trained it on some data (about 20000 images with 20 classes). I set the number of steps to 25000 and batch size to 3 (...
- 103
2
votes
1
answer
3k
views
Does BERT freeze the entire model body when it does fine-tuning?
Recently, I came across the BERT model. I did some research and tried some implementations.
I wanted to tackle a NER task, so I chose the BertForSequenceClassifications provided by HuggingFace.
...
- 51
1
vote
1
answer
96
views
Would this count as a Transfer Learning approach?
I have two datasets, Dataset 1(D1) and Dataset 2(D2). D1 has around 22000 samples, and D2 has around 8000 samples. What I am doing is that I train a Deep Neural Network model with around three layers ...
- 133
2
votes
2
answers
3k
views
What is the difference between feature extraction and fine-tuning in transfer learning?
I'm building a model for facial expression recognition, and I want to use transfer learning. From what I understand, there are different steps to do it. The first is the feature extraction and the ...
- 21
1
vote
1
answer
248
views
Can Facebook's LASER be used like BERT?
Can Facebook's LASER be fine-tuned like BERT for Question Answering tasks or Sentiment Analysis?
From my understanding, they created an embedding that allows for similar words in different languages ...
- 15
2
votes
0
answers
38
views
Adding corpus to BERT for QA
I was wondering about SciBERT's QA abilities using SQuAD. I have a scarce textual dataset consisting of less than 100 files where doctors are discussing cancer in dialogues. I want to add it to ...
- 121
1
vote
0
answers
54
views
How to scrape product data on supplier websites?
I'm currently trying to build a semantic scraper that can extract product information from different company websites of suppliers in the packaging industry (with as little manual customization per ...
- 11
1
vote
0
answers
53
views
What is the definition of pre-training?
I want to pre-train a model (combined by two popular modules A and B, and both are large blocks), then fine-tune it on downstream tasks.
What if for the weight initialization for pre-training, module ...
- 11
3
votes
0
answers
117
views
Why shouldn't batch normalisation layers be learnable during fine-tuning?
I have been reading this TensorFlow tutorial on transfer learning, where they unfroze the whole model and then they say:
When you unfreeze a model that contains ...
- 862
0
votes
0
answers
83
views
Is it possible to improve the average precision of YOLO trained on Open Images Dataset by fine-tuning it with COCO?
I consider pre-training a YOLOv5 with Google Open Images Object Detection dataset. The dataset includes general domain categories with ~15 M box samples. After the pre-training is done, I will fine-...
- 101
4
votes
1
answer
2k
views
Why aren't the BERT layers frozen during fine-tuning tasks?
During transfer learning in computer vision, I've seen that the layers of the base model are frozen if the images aren't too different from the model on which the base model is trained on.
However, on ...
- 141
0
votes
1
answer
62
views
BERT: After pretraining 880000 step, why fine-tune not work? [closed]
I am using pretraining code from https://github.com/NVIDIA/DeepLearningExamples
Pretrain parameters:
...
- 101
4
votes
1
answer
4k
views
How to fine tune BERT for question answering?
I wish to train two domain-specific models:
Domain 1: Constitution and related Legal Documents
Domain 2: Technical and related documents.
For Domain 1, I've access to a text-corpus with texts from ...
- 141
12
votes
1
answer
4k
views
What is the difference between one-shot learning, transfer learning and fine tuning?
Lately, there are lots of posts on one-shot learning. I tried to figure out what it is by reading some articles. To me, it looks like similar to transfer learning, in which we can use pre-trained ...
- 406
0
votes
1
answer
4k
views
GPT-2: (Hardware) requirements for fine-tuning the 774M model [closed]
I wonder if there's anyone who has actually succeeded in fine-tuning GPT-2's 774M model without using cloud TPU's. My GeForce RTX 2070 SUPER couldn't handle it in previous attempts.
I'm running ...
- 129