Skip to main content

Questions tagged [fine-tuning]

For questions related to the concept of fine-tuning a model (e.g. neural network), which is very related to and sometimes used as a synonym for transfer learning.

Filter by
Sorted by
Tagged with
0 votes
0 answers
5 views

QLora using peft from HF and custom class for binary classification

I am fine-tuning an mistral-7B LLM model for binary classification. I realize it may be an overkill; but we are running some experiments. So far, I have used HuggingFace libraries like peft and ...
kms's user avatar
  • 101
0 votes
0 answers
19 views

Does the type of saving the model affects the training result? [closed]

I saved the pretrained network in two ways 1. ...
COTHE's user avatar
  • 11
0 votes
1 answer
25 views

Fintuning Time of LoRA

I tried lora (on ViT) and i thought it would reduce the finetuning time, but it is same. is it right?? I checked that the original network's weights are forzen correctly. Then LoRA's advantage is ...
COTHE's user avatar
  • 11
0 votes
0 answers
13 views

Unexpected results using ORPO trl

For studying purposes, I've created a very small dataset about a fictional city called "Auryn": https://huggingface.co/datasets/celsowm/auryn_dpo_orpo_english So, my goal is to "inject&...
celsowm's user avatar
  • 101
0 votes
1 answer
30 views

Why does LoRA has encoder-decoder structure?

I've learned that LoRA injects trainable rank decomposition 'matrices' into each layer. A matrix reduces dimension from d to r, and B matrix increases dimension from r to d. What I'm having trouble in ...
COTHE's user avatar
  • 11
2 votes
1 answer
59 views

Why fine tuning does not work as well as RAG?

I cannot find a definite answer to this question. Suppose I want to build a QA (question answering) system on a set of personal documents. It looks that RAG (retrieval augmented generation) is the way ...
Thomas's user avatar
  • 193
2 votes
1 answer
52 views

How is a LLM able to override its prior knowledge through In-Context Learning?

I came across a Google's blog (https://research.google/blog/larger-language-models-do-in-context-learning-differently/) discussing large language models (LLMs) and how we can overried their prior ...
tidealwari91's user avatar
1 vote
1 answer
90 views

Can you finetune an LLM using negative examples?

If I have a negative example from a large language model like ...
tSchema's user avatar
  • 111
0 votes
0 answers
13 views

Unable to Save Generated Data to JSONL File - Always Resulting in "Wrote 0 examples to finetuning_events.jsonl" Message

Issue Description When attempting to generate JSONL data using Llama Index, the process works well until the final step where the results are saved to a JSONL file. However, every time I try to save ...
gluck0101's user avatar
  • 101
0 votes
0 answers
48 views

How to teach Gemma model my mother tongue (Kannada - one of the oldest Indic languages)

I'm interested in teaching the Gemma 2B model my mother tongue (Kannada - one of the oldest Indic languages). The pre-trained model doesn't work well with the mentioned language, so I thought of ...
Swastik's user avatar
  • 101
0 votes
0 answers
12 views

What effect is expected if LoRA is applied after Fine-tuning?

I am currently learning several things about the ASR Transformer model. Recently, I learned LoRA and Adapter. It certainly seems to have an advantage over fine-tuning in general. But here I came up ...
C yp's user avatar
  • 1
1 vote
1 answer
25 views

Multi-task objective sometimes improve single-task performance, but is this true when fine tuning?

It is known that multitask objectives in neural networks sometimes have the effect of improving the performance of the neural network for each of the tasks individually (versus training the same ...
Alexander Soare's user avatar
0 votes
0 answers
25 views

Can we use quantization techniques to train ML models from scratch?

I'm currently learning about quantization and have seen that most methods for quantizing are for a Post-Training conversion or, in some cases, for efficient finetuning like the QLORA method. I'd like ...
Cesar Ruiz's user avatar
1 vote
0 answers
71 views

AI chat bot that answers by focusing only on 30 textbooks [closed]

I don't even know what I'm looking for and what's the terminology, so here I am asking this question. Background Assume I have 30 textbooks. I want to have an AI chatbot like ChatGPT which answers the ...
Megidd's user avatar
  • 141
0 votes
0 answers
57 views

Does DPO update the weights in the same way that LORA/Fine-Tuining does?

I've been working on making a conversational customer service fine-tune for the past couple months and now I am looking to improve its failure recovery. For example, If the agent makes a mistake, how ...
the-test-set-is-all-you-need's user avatar
0 votes
0 answers
46 views

Eval loss when fine-tuning in an unsupervised way/pretraining?

I'm fine-tuning the base Mixtral 8x7B model (4-bit quantized) with Lora on my own data, following these guidelines: https://www.stochastic.ai/blog/xfinance-vs-bloomberg-gpt I'm first fine-tuning it in ...
Jon Flynn's user avatar
  • 101
0 votes
0 answers
22 views

Does it make sense to use static masks to improve the quality of a zero-shot source separation model

I'm thinking about ways to fine-tune a neural network for source separation, specifically AudioSep It is known that in nlp tasks, RAG is often used as an alternative to fine tuning to provide LLM with ...
araxal's user avatar
  • 1
0 votes
0 answers
35 views

Any suggestions for transformer finetuning techniques ablation study?

I'm planning to fine tune a 7b parameter model for a research project. I understand the different steps of model fine tuning, namely Supervised fine tuning - where we train model on curated examples ...
kaiser's user avatar
  • 101
0 votes
0 answers
58 views

What are some tips of curating a dataset to fine-tune a code-completion LLM?

There is a new SDK that I am working on and I want to know what are some ways of automatically curating a dataset to train a code-completing LLM to deploy as a VSCode plugin? Hacky ways are ...
Levent Ozbek's user avatar
1 vote
0 answers
652 views

Finetuning Mistral or MistralForSequenceClassification for text classification

I need to do text classification and have a dataset of 10K entries. I am considering using mistral and following a tutorial like https://huggingface.co/docs/transformers/training and replace model ...
Karl 17302's user avatar
1 vote
0 answers
17 views

Looking for a way to train a model to learn optimal parameters/hyperparameters of clustering

I have 5000 docs, each is a review. For each review, i'm plotting the sentences in a semantic dimension. Now, I'm applying clustering to these points for each review. The success of my model depends ...
Prithvi's user avatar
  • 11
0 votes
0 answers
50 views

When using Reinforcement Learning with Human Feedback to train a transformer, how do I propagate the feedback through the transformer?

I'm basically trying to replicate the processed used to create Chat GPT: Am I supposed to backpropagate? How can I do that when these aren't really errors, but rather ranking several response? Can I ...
Austin Capobianco's user avatar
0 votes
0 answers
4 views

Individual Data point size within a dataset for finetuning for translation

I need to fine tuning a LLM for a translation task for specific domain. Could someone please advise if length of each data point within the data set has any impact on eventual performance of the LLMs. ...
user11740597's user avatar
0 votes
0 answers
23 views

Discussion about Improving Visual Search Model Accuracy

My Visual Search Model is only achieving an accuracy of about 42% If anyone can give me advice to drastically improve this number I would greatly appreciate it. Below is my current flow of image ...
rileylivingston's user avatar
0 votes
0 answers
30 views

Training YOLO on YOLO results

I have a large unlabelled dataset. What if I use YOLO to label it? Will this dataset be useful to train a better YOLO model? What if I then finetune it on a smaller labelled dataset? My usecase ...
Karol Idaszak's user avatar
2 votes
0 answers
153 views

What type of fine-tuning does the OpenAI API use?

They have instructions on how to fine-tune. But I'm not clear on the type of fine-tuning that is done. Maybe this is mentioned somewhere. I would imagine it's some type of PEFT to avoid catastrophic ...
user14094230's user avatar
2 votes
2 answers
3k views

How can I teach a book to an LLM?

I am trying to find out how I can teach the content of a whole, multiple hundert pages book to an LLM so that it "knows" all details and can be queried, give summaries etc. The book is one ...
dschuld's user avatar
  • 21
0 votes
0 answers
44 views

Any research in "probe-tuning" of LLMs?

Is there any research in "probe-tuning" of LLMs, i.e., tuning LLM's parameter weights such that a specific probe (classifier) is more reliably detecting certain markers throughout the ...
leventov's user avatar
  • 101
0 votes
0 answers
54 views

can I finetune a model on Azure for information extraction based on "question", "context", and "answer" training data?

I am working on extracting certain fields from a large corpus. I was looking at finetuning an LLM on Azure for the task. I think finetuning is the right idea (as opposed to vector databases, or RAG), ...
trailer_swift's user avatar
0 votes
0 answers
336 views

Optimal Quantity of Training Data for Fine-Tuning an LLM: Is Bigger Always Better?

I am currently working on fine-tuning an LLM for a specific task, and I am trying to determine the optimal size for my training dataset. Intuitively, one might think that the more data, the better. ...
Peyman's user avatar
  • 574
2 votes
2 answers
563 views

What is the difference betwen fine runing and rlhf for llm?

I am confused about the difference betwen fine runing and rlhf for llm. When to use what? I know RLHF need to creating a reward model which at furst rates responses to align the responses to the ...
Exploring's user avatar
  • 353
0 votes
0 answers
31 views

is it possible to make use of classes of coco-pretrained weights on custom dataset just training on custom dataset?

I am trying to use detcetron2 panoptic_FPN and panoptic_deeplab models for optimization on custom dataset. You might already now that coco has around 133 classes (both thing and stuff). And my custom ...
srikanth reddy's user avatar
0 votes
2 answers
131 views

Does fine-tuning a multilingual transformer model allow it to generalize to languages unseen in the fine-tuning dataset?

Example: https://huggingface.co/google/umt5-base Note: UMT5 was only pre-trained on mC4 excluding any supervised training. Therefore, this model has to be fine-tuned before it is useable on a ...
Michał B.'s user avatar
0 votes
0 answers
183 views

How to fine-tune a pre-trained model for customer service like tasks chatbot?

I want to make a trainable conversational bot that can respond to customer service questions. The bot should be able to adapt to different domains based on the dataset provided by the user (a company ...
Mohamed saad's user avatar
0 votes
1 answer
143 views

Fine-tuning or Prompt Engineering or both?

We have a dataset of legal cases, academic papers etc which we will load into a vector database. We want to develop an agent that will allow a user to enter a specific legal issue, the agent then will ...
GEM's user avatar
  • 1
3 votes
1 answer
3k views

Should I be layer freezing when fine-tuning an LLM?

I've had it in my head that generally speaking, it's better to freeze layers when fine-tuning an LLM, as per this quote from HuggingFace's article: PEFT approaches only fine-tune a small number of (...
multiheadedattention's user avatar
1 vote
0 answers
509 views

fine-tune nanoGPT for instructions

I've been playing around with nanoGPT, and recently I decided I wanted to fine-tune it using the dolly instruction set. This data set consists of roughly 15k examples and each example has the ...
CWcx's user avatar
  • 111
1 vote
1 answer
319 views

What researched-backed findings is there for prompting LLM’s / GPT-4 to give specific information or actionable plans?

I have learned a bit recently about prompt strategies. For example, there was a paper about how just by saying “Let’s think step by step” can increase answer quality by like 40%. I have also come to ...
Julius Hamilton's user avatar
1 vote
1 answer
187 views

Teaching an LLM about daily updated machine-readable information so it can respond questions

I’m quite new in this field, and despite having spent some good amount of time learning the ins and outs of frameworks like LangChain, and browsing around the internet quite a bit, I still don’t know ...
newlog's user avatar
  • 121
1 vote
2 answers
158 views

4 Questions on Transformers [closed]

Assume the transformer is trained on 512 max length sentences: can we fine-tune it on 256 max-length sentences? If we can fine-tune it, how is it even possible because the input shapes are different,...
Arjun Reddy's user avatar
0 votes
1 answer
106 views

Fine-Tuning T5 with specific penalty

Currently I am finetuning transformers T5 model for translation task. As part of the dataset, I am given sentences in Japanese, their translation to English, and ...
Dani's user avatar
  • 131
2 votes
1 answer
742 views

Creating a support chat bot for my business

I am trying to create a kind of support bot to answer questions from my clients about specific technical details about WordPress plugins that I sell. The goal is that the ...
digitalzoomstudio's user avatar
0 votes
0 answers
301 views

When would you use prompting vs. fine tuning?

I would like to hear your thoughts on when is it appropriate to use prompting vs. fine-tuning. Does one make more sense for specific tasks than the other. Kindly elaborate.
Josiah Cheruiyot's user avatar
1 vote
2 answers
704 views

Does layer freezing offer other benefits other than to reduce computational time in gradient descent?

In Deep Learning and Transfer Learning, does layer freezing offer other benefits other than to reduce computational time in gradient descent? Assuming I train a neural network on task A to derive ...
Enk9456's user avatar
  • 21
1 vote
1 answer
160 views

What background should I have before starting to fine tune a Large Language Model?

I want to know what things I should be learning before trying to fine-tune or for that matter working with a large language model. In my case, I am trying to fine-tune bloom (https://huggingface.co/...
kelly's user avatar
  • 23
10 votes
2 answers
3k views

Are GPT-3.5 series models based on GPT-3?

In the official blog post about ChatGPT from OpenAI, there is this paragraph explaining how ChatGPT model was trained: We trained this model using Reinforcement Learning from Human Feedback (RLHF), ...
iMad's user avatar
  • 203
2 votes
1 answer
158 views

Would a transformer trained on highly specific material be as usable as a commercial product like ChatGPT?

Soft question here. I was recently learning a bit about how it is feasible to train a transformer on a personal computer like an M1 Mac. I have been told that the model could have 1-3 million ...
Julius Hamilton's user avatar
1 vote
1 answer
629 views

For specific tasks, is it better to fine-tune models on examples or just use prompting with the context of the task?

These days large language models cover a vast amount of topics and information, but I wanted to understand: For specific tasks, is it better to fine-tune models on examples or just use prompting with ...
Imran Q's user avatar
  • 113
1 vote
1 answer
3k views

Fine-tune GPT-Neo with prompt and completion?

I'm new to AI and machine learning. To fine-tune GPT-3, I understand that we need a set of training examples that each consist of a single input ("prompt") and its associated output ("...
SoftTimur's user avatar
  • 111
3 votes
0 answers
843 views

What is the difference between prompt tuning and prefix tuning?

I read prompt tuning and prefix tuning are two effective mechanisms to leverage frozen language models to perform downstream tasks. What is the difference between the two and how they work really? ...
Exploring's user avatar
  • 353