Questions tagged [fine-tuning]

For questions related to the concept of fine-tuning a model (e.g. neural network), which is very related to and sometimes used as a synonym for transfer learning.

Filter by
Sorted by
Tagged with
0 votes
0 answers
21 views

Does DPO update the weights in the same way that LORA/Fine-Tuining does?

I've been working on making a conversational customer service fine-tune for the past couple months and now I am looking to improve its failure recovery. For example, If the agent makes a mistake, how ...
the-test-set-is-all-you-need's user avatar
0 votes
1 answer
22 views

Fine-tuning pretrained YOLOv7 with new data without affecting existing accuracy [closed]

I have a pre-trained YOLOv7-base(NOT tiny) pt file, but i do not have access to the dataset using which it was trained. I want to improve the performance(detection and confidence) for certain ...
9friday's user avatar
0 votes
0 answers
30 views

Eval loss when fine-tuning in an unsupervised way/pretraining?

I'm fine-tuning the base Mixtral 8x7B model (4-bit quantized) with Lora on my own data, following these guidelines: https://www.stochastic.ai/blog/xfinance-vs-bloomberg-gpt I'm first fine-tuning it in ...
Jon Flynn's user avatar
  • 101
0 votes
0 answers
21 views

Does it make sense to use static masks to improve the quality of a zero-shot source separation model

I'm thinking about ways to fine-tune a neural network for source separation, specifically AudioSep It is known that in nlp tasks, RAG is often used as an alternative to fine tuning to provide LLM with ...
araxal's user avatar
  • 1
0 votes
0 answers
23 views

Any suggestions for transformer finetuning techniques ablation study?

I'm planning to fine tune a 7b parameter model for a research project. I understand the different steps of model fine tuning, namely Supervised fine tuning - where we train model on curated examples ...
kaiser's user avatar
  • 101
0 votes
0 answers
21 views

What are some tips of curating a dataset to fine-tune a code-completion LLM?

There is a new SDK that I am working on and I want to know what are some ways of automatically curating a dataset to train a code-completing LLM to deploy as a VSCode plugin? Hacky ways are ...
Onur-Andros Ozbek's user avatar
0 votes
0 answers
367 views

Finetuning Mistral or MistralForSequenceClassification for text classification

I need to do text classification and have a dataset of 10K entries. I am considering using mistral and following a tutorial like https://huggingface.co/docs/transformers/training and replace model ...
Karl 17302's user avatar
1 vote
0 answers
17 views

Looking for a way to train a model to learn optimal parameters/hyperparameters of clustering

I have 5000 docs, each is a review. For each review, i'm plotting the sentences in a semantic dimension. Now, I'm applying clustering to these points for each review. The success of my model depends ...
Prithvi's user avatar
  • 11
0 votes
0 answers
21 views

When using Reinforcement Learning with Human Feedback to train a transformer, how do I propagate the feedback through the transformer?

I'm basically trying to replicate the processed used to create Chat GPT: Am I supposed to backpropagate? How can I do that when these aren't really errors, but rather ranking several response? Can I ...
Austin Capobianco's user avatar
0 votes
0 answers
4 views

Individual Data point size within a dataset for finetuning for translation

I need to fine tuning a LLM for a translation task for specific domain. Could someone please advise if length of each data point within the data set has any impact on eventual performance of the LLMs. ...
user11740597's user avatar
0 votes
0 answers
22 views

Discussion about Improving Visual Search Model Accuracy

My Visual Search Model is only achieving an accuracy of about 42% If anyone can give me advice to drastically improve this number I would greatly appreciate it. Below is my current flow of image ...
rileylivingston's user avatar
0 votes
0 answers
20 views

Training YOLO on YOLO results

I have a large unlabelled dataset. What if I use YOLO to label it? Will this dataset be useful to train a better YOLO model? What if I then finetune it on a smaller labelled dataset? My usecase ...
Karol Idaszak's user avatar
2 votes
0 answers
147 views

What type of fine-tuning does the OpenAI API use?

They have instructions on how to fine-tune. But I'm not clear on the type of fine-tuning that is done. Maybe this is mentioned somewhere. I would imagine it's some type of PEFT to avoid catastrophic ...
user14094230's user avatar
1 vote
2 answers
953 views

How can I teach a book to an LLM?

I am trying to find out how I can teach the content of a whole, multiple hundert pages book to an LLM so that it "knows" all details and can be queried, give summaries etc. The book is one ...
dschuld's user avatar
  • 11
0 votes
0 answers
31 views

Any research in "probe-tuning" of LLMs?

Is there any research in "probe-tuning" of LLMs, i.e., tuning LLM's parameter weights such that a specific probe (classifier) is more reliably detecting certain markers throughout the ...
leventov's user avatar
  • 101
0 votes
0 answers
42 views

can I finetune a model on Azure for information extraction based on "question", "context", and "answer" training data?

I am working on extracting certain fields from a large corpus. I was looking at finetuning an LLM on Azure for the task. I think finetuning is the right idea (as opposed to vector databases, or RAG), ...
trailer_swift's user avatar
0 votes
0 answers
321 views

Optimal Quantity of Training Data for Fine-Tuning an LLM: Is Bigger Always Better?

I am currently working on fine-tuning an LLM for a specific task, and I am trying to determine the optimal size for my training dataset. Intuitively, one might think that the more data, the better. ...
Peyman's user avatar
  • 564
2 votes
2 answers
349 views

What is the difference betwen fine runing and rlhf for llm?

I am confused about the difference betwen fine runing and rlhf for llm. When to use what? I know RLHF need to creating a reward model which at furst rates responses to align the responses to the ...
Exploring's user avatar
  • 321
0 votes
0 answers
27 views

is it possible to make use of classes of coco-pretrained weights on custom dataset just training on custom dataset?

I am trying to use detcetron2 panoptic_FPN and panoptic_deeplab models for optimization on custom dataset. You might already now that coco has around 133 classes (both thing and stuff). And my custom ...
srikanth reddy's user avatar
0 votes
2 answers
110 views

Does fine-tuning a multilingual transformer model allow it to generalize to languages unseen in the fine-tuning dataset?

Example: https://huggingface.co/google/umt5-base Note: UMT5 was only pre-trained on mC4 excluding any supervised training. Therefore, this model has to be fine-tuned before it is useable on a ...
Michał B.'s user avatar
0 votes
0 answers
152 views

How to fine-tune a pre-trained model for customer service like tasks chatbot?

I want to make a trainable conversational bot that can respond to customer service questions. The bot should be able to adapt to different domains based on the dataset provided by the user (a company ...
Mohamed saad's user avatar
0 votes
0 answers
18 views

How to extract body of a base-model and fine tune with that body on different shape dataset like this situation

In BERT like transformer model (I am not talking about BERT in this thread), it has 2 training objectives Masked Language Modeling and Next sentence prediction right? and BERT model is also supports ...
Arjun Reddy's user avatar
0 votes
1 answer
134 views

Fine-tuning or Prompt Engineering or both?

We have a dataset of legal cases, academic papers etc which we will load into a vector database. We want to develop an agent that will allow a user to enter a specific legal issue, the agent then will ...
GEM's user avatar
  • 1
3 votes
1 answer
2k views

Should I be layer freezing when fine-tuning an LLM?

I've had it in my head that generally speaking, it's better to freeze layers when fine-tuning an LLM, as per this quote from HuggingFace's article: PEFT approaches only fine-tune a small number of (...
multiheadedattention's user avatar
1 vote
0 answers
422 views

fine-tune nanoGPT for instructions

I've been playing around with nanoGPT, and recently I decided I wanted to fine-tune it using the dolly instruction set. This data set consists of roughly 15k examples and each example has the ...
CWcx's user avatar
  • 111
1 vote
1 answer
269 views

What researched-backed findings is there for prompting LLM’s / GPT-4 to give specific information or actionable plans?

I have learned a bit recently about prompt strategies. For example, there was a paper about how just by saying “Let’s think step by step” can increase answer quality by like 40%. I have also come to ...
Julius H.'s user avatar
  • 225
1 vote
1 answer
166 views

Teaching an LLM about daily updated machine-readable information so it can respond questions

I’m quite new in this field, and despite having spent some good amount of time learning the ins and outs of frameworks like LangChain, and browsing around the internet quite a bit, I still don’t know ...
newlog's user avatar
  • 121
1 vote
2 answers
149 views

4 Questions on Transformers [closed]

Assume the transformer is trained on 512 max length sentences: can we fine-tune it on 256 max-length sentences? If we can fine-tune it, how is it even possible because the input shapes are different,...
Arjun Reddy's user avatar
0 votes
1 answer
92 views

Fine-Tuning T5 with specific penalty

Currently I am finetuning transformers T5 model for translation task. As part of the dataset, I am given sentences in Japanese, their translation to English, and ...
Dani's user avatar
  • 121
2 votes
1 answer
563 views

Creating a support chat bot for my business

I am trying to create a kind of Support Bot to answer my clients about specific technical details about WordPress plugins that I sell. The goal is that the /completitions api would be feeded a prompt ...
digitalzoomstudio's user avatar
0 votes
0 answers
292 views

When would you use prompting vs. fine tuning?

I would like to hear your thoughts on when is it appropriate to use prompting vs. fine-tuning. Does one make more sense for specific tasks than the other. Kindly elaborate.
Josiah Cheruiyot's user avatar
1 vote
2 answers
557 views

Does layer freezing offer other benefits other than to reduce computational time in gradient descent?

In Deep Learning and Transfer Learning, does layer freezing offer other benefits other than to reduce computational time in gradient descent? Assuming I train a neural network on task A to derive ...
Enk9456's user avatar
  • 21
1 vote
1 answer
153 views

What background should I have before starting to fine tune a Large Language Model?

I want to know what things I should be learning before trying to fine-tune or for that matter working with a large language model. In my case, I am trying to fine-tune bloom (https://huggingface.co/...
kelly's user avatar
  • 23
10 votes
2 answers
3k views

Are GPT-3.5 series models based on GPT-3?

In the official blog post about ChatGPT from OpenAI, there is this paragraph explaining how ChatGPT model was trained: We trained this model using Reinforcement Learning from Human Feedback (RLHF), ...
iMad's user avatar
  • 203
2 votes
1 answer
144 views

Would a transformer trained on highly specific material be as usable as a commercial product like ChatGPT?

Soft question here. I was recently learning a bit about how it is feasible to train a transformer on a personal computer like an M1 Mac. I have been told that the model could have 1-3 million ...
Julius H.'s user avatar
  • 225
1 vote
1 answer
578 views

For specific tasks, is it better to fine-tune models on examples or just use prompting with the context of the task?

These days large language models cover a vast amount of topics and information, but I wanted to understand: For specific tasks, is it better to fine-tune models on examples or just use prompting with ...
Imran Q's user avatar
  • 113
1 vote
1 answer
3k views

Fine-tune GPT-Neo with prompt and completion?

I'm new to AI and machine learning. To fine-tune GPT-3, I understand that we need a set of training examples that each consist of a single input ("prompt") and its associated output ("...
SoftTimur's user avatar
  • 111
3 votes
0 answers
822 views

What is the difference between prompt tuning and prefix tuning?

I read prompt tuning and prefix tuning are two effective mechanisms to leverage frozen language models to perform downstream tasks. What is the difference between the two and how they work really? ...
Exploring's user avatar
  • 321
1 vote
0 answers
23 views

How can (pretrained) language models actively seek additional training data - possibly reference request?

I am reading the paper "Large Language Models Can Self-Improve" https://arxiv.org/abs/2210.11610 in which the authors consider that LLM can generate Chain-of-Thoughts sequences and even ...
TomR's user avatar
  • 843
1 vote
1 answer
44 views

Fine tuning a Deep Learning model post training

I have trained a CNN in a binary classification problem, however the original problem has 6 different classes, of which, I am only interested in classifying one, so if it is that certain class or not....
NeuroEng's user avatar
  • 121
0 votes
1 answer
47 views

can I add to a language model a prompt with output example?

I want to finetune GPT2 to extract relevant data from a given text. So for (a trivial) example, given the text "the car was manufactured in X, can reach Y km/h, and has Z horse powers", my ...
Hadar Sharvit's user avatar
0 votes
2 answers
988 views

How to combine pretrained language models with additional feature set?

Are there any techniques to combine a feature set (other than the text itself) with pretrained language models. Let's say I have a random NLP task that tries to predict a binary class label based on e....
fragant's user avatar
  • 101
4 votes
1 answer
4k views

What is the difference between fine tuning and variants of few shot learning? [duplicate]

I am trying to understand the concept of fine-tuning and few-shot learning. I understand the need for fine-tuning. It is essentially tuning a pre-trained model to a specific downstream task. However, ...
Exploring's user avatar
  • 321
0 votes
2 answers
5k views

How to fine-tune GPT-J with small dataset

I have followed this guide as closely as possible: https://github.com/kingoflolz/mesh-transformer-jax I'm trying to fine-tune GPT-J with a small dataset of ~500 lines: ...
Ilya Karnaukhov's user avatar
1 vote
0 answers
123 views

How to fine-tune a model which was pre-trained on a corpus that contains words with different meanings than the meanings of those words on my corpus?

I have a scenario in which we should leverage previously asked questions (not questions pairs, single question in a column) to locate similar questions within those questions. How can I fine-tune my ...
smanem's user avatar
  • 11
0 votes
1 answer
1k views

Is it possible that the fine-tuned pre-trained model performs worse than the original pre-trained model?

I have downloaded a pre-trained EfficientDet D2 model (Tensorflow 2.0) and trained it on some data (about 20000 images with 20 classes). I set the number of steps to 25000 and batch size to 3 (...
Araw's user avatar
  • 103
2 votes
1 answer
3k views

Does BERT freeze the entire model body when it does fine-tuning?

Recently, I came across the BERT model. I did some research and tried some implementations. I wanted to tackle a NER task, so I chose the BertForSequenceClassifications provided by HuggingFace. ...
Joon's user avatar
  • 51
1 vote
1 answer
99 views

Would this count as a Transfer Learning approach?

I have two datasets, Dataset 1(D1) and Dataset 2(D2). D1 has around 22000 samples, and D2 has around 8000 samples. What I am doing is that I train a Deep Neural Network model with around three layers ...
Ravish Jha's user avatar
2 votes
2 answers
4k views

What is the difference between feature extraction and fine-tuning in transfer learning?

I'm building a model for facial expression recognition, and I want to use transfer learning. From what I understand, there are different steps to do it. The first is the feature extraction and the ...
Speedskillsx's user avatar
2 votes
1 answer
265 views

Can Facebook's LASER be used like BERT?

Can Facebook's LASER be fine-tuned like BERT for Question Answering tasks or Sentiment Analysis? From my understanding, they created an embedding that allows for similar words in different languages ...
user1234's user avatar