13 votes
Accepted

Are GPT-3.5 series models based on GPT-3?

I'll complement nbro's answer with this great visual summary by Yao Fu <[email protected]>:
Franck Dernoncourt's user avatar
7 votes

Are GPT-3.5 series models based on GPT-3?

ChatGPT has not been trained from scratch. ChatGPT is a fine-tuned version of a model from the GPT-3.5 series. OpenAI writes ChatGPT is fine-tuned from a model in the GPT-3.5 series, which finished ...
nbro's user avatar
  • 40.5k
7 votes
Accepted

What is the difference between one-shot learning, transfer learning and fine tuning?

They are all related terms. From top to bottom: One-shot learning aims to achieve results with one or very few examples. Imagine an image classification task. You may show an apple and a knife to a ...
Pablo's user avatar
  • 226
4 votes

Creating a support chat bot for my business

First and foremost, do not use GPT/OpenAI for customer-facing applications. You end up with a mess. GPT is great for creative work, but not for production. GPT is a probabilistic language model, and ...
kokumajutsu's user avatar
4 votes
Accepted

Is it possible that the fine-tuned pre-trained model performs worse than the original pre-trained model?

Yes, this is quite the expected behavior. The main difference between expected and current behavior lies in the amount of data you are using for training VS the amount of data that the pre-trained ...
JVGD's user avatar
  • 1,108
3 votes
Accepted

Does BERT freeze the entire model body when it does fine-tuning?

Taken directly from HuggingFace Note that if you are used to freezing the body of your pretrained model (like in computer vision) the above may seem a bit strange, as we are directly fine-tuning the ...
Joon's user avatar
  • 51
3 votes
Accepted

What background should I have before starting to fine tune a Large Language Model?

There are considerable free and excellent resources out there (in the wild). You can check The Stanford Natural Language Processing Group teaching page; you can easily follow their YouTube courses on ...
Eduard's user avatar
  • 211
3 votes

What is the difference between fine tuning and variants of few shot learning?

I believe the standard meanings are as follows, but not everyone uses words in the same way, so you might see examples that differ. Fine tuning refers to slightly changing the weights of a pre-trained ...
Lee Reeves's user avatar
2 votes
Accepted

Would a transformer trained on highly specific material be as usable as a commercial product like ChatGPT?

Yes, with caveats. Yes: If the data covers a niche and is very rare, you can indeed fine-tune a large model to your needs. Caveats: Fine tuning a model still require significant compute. Moreover the ...
Rexcirus's user avatar
  • 1,174
2 votes
Accepted

For specific tasks, is it better to fine-tune models on examples or just use prompting with the context of the task?

Fine tuning is superior, since the whole network specialise to solve a given problem only. A specialist will always beat a generalist in the specialised task. That said, if the generalist network is ...
Rexcirus's user avatar
  • 1,174
2 votes

What is the difference betwen fine runing and rlhf for llm?

RLHF is just one possibility of fine-tuning for generative LLMs, which is used to align an LLM to human tastes. However, you could just create a bunch of great data, and fine-tune (take a pretrained ...
Alberto's user avatar
  • 1,905
2 votes
Accepted

Does fine-tuning a multilingual transformer model allow it to generalize to languages unseen in the fine-tuning dataset?

The short answer: Very unlikely. The extended answer: If you fine-tune a model, it becomes specialized for the type of data you fine-tune it on but you trade in some of its generalization capabilities....
emely_pi's user avatar
  • 277
1 vote

How can I teach a book to an LLM?

Using embeddings is an effective approach when you have limited data, such as a book, and want to extract relevant context and related text for querying. An example of this approach at this URL: https:...
Harsh Gill's user avatar
1 vote

Does layer freezing offer other benefits other than to reduce computational time in gradient descent?

Both are transfer learning approaches, which this Pytorch tutorial explains very well: https://pytorch.org/tutorials/beginner/transfer_learning_tutorial.html In practice, very few people train an ...
Rexcirus's user avatar
  • 1,174
1 vote

Does layer freezing offer other benefits other than to reduce computational time in gradient descent?

from your post I assume you are having three sub-questions and I will answer them one by one. For the 1st question, yes, layer freezing reduces the computational cost a lot and also helps the model ...
GunFire's user avatar
  • 124
1 vote

Does fine-tuning a multilingual transformer model allow it to generalize to languages unseen in the fine-tuning dataset?

The attention doesn't discriminate between what token it is producing as long as it is following the protocol/heuristic defined by the finetuning dataset, So essentially, a finetuning dataset that ...
Chinmay's user avatar
  • 521
1 vote

Fine-tuning or Prompt Engineering or both?

I am completing a master's degree in artificial intelligence.I am researching this field. Initially, try training using techniques like QLora, which was released this week. test distillation ...
Alicia Colmenero-Fernandez's user avatar
1 vote

What researched-backed findings is there for prompting LLM’s / GPT-4 to give specific information or actionable plans?

It may not be a complete answer to your question as it is quite broad, but I would offer three approaches for you to have a look at: Fine-tuning via JSON: Ranedeer. Mr. Ranedeer AI Tutor is a ...
Hans-Peter Schrei's user avatar
1 vote

Teaching an LLM about daily updated machine-readable information so it can respond questions

I was able to create a first implementation of this using a LangChain agent that dynamically queries a dataset, spits Python, and executes it. All of this can be done with, for example, the CSV Agent ...
newlog's user avatar
  • 121
1 vote

4 Questions on Transformers

You can implement the Transformer architecture such that you can indeed change the block size. It can be made smaller and larger. It should be noted, however, that there is no guarantee on performance ...
Robin van Hoorn's user avatar
1 vote

Fine-Tuning T5 with specific penalty

Actually I encountered this issue myself. My solution was to wrap the model in my own class. I then overrode the forward method. ( I was using pytorch lightning) ...
Thomas K's user avatar
1 vote

GPT-2: (Hardware) requirements for fine-tuning the 774M model

Possibly a bit late to the answer, but I doubt you'd be able to run GPT-2 774M in FP32 on 2070 Super which has 8GB VRAM. I know it's not an exact comparison, but fine-tuning BERT Large (345M) in FP32 ...
Dan Pavlov's user avatar
1 vote
Accepted

When doing transfer learning, which initial layers do we need to freeze, and how should I change the last layer for my task?

After a lot of browsing online for answers to these questions, this is what I came up with. What are the initial layers in this case? How exactly can I freeze them? The initial few layers are said ...
Santhosh's user avatar
  • 143
1 vote
Accepted

Would this count as a Transfer Learning approach?

This tells me that because the initial model was performing so poorly without any fine-tuning, most of the learning that led to the 90% accuracy was only because of the additional layers, not the ...
Neil Slater's user avatar
  • 32.1k
1 vote

What is the difference between feature extraction and fine-tuning in transfer learning?

The difference between the two approaches (feature extraction vs fine-tuning) is well explained here: Fine Tuning vs Joint Training vs Feature Extraction Also, this paper evaluate the performance one ...
couturierc's user avatar
1 vote

Can Facebook's LASER be used like BERT?

LASER creates multilingual contextualized word embeddings, what you do with them is up to you. You can use this as a feature extraction and add whatever you want to the end of the network. I believe ...
Isbister's user avatar
  • 186
1 vote

How to fine tune BERT for question answering?

The answer is yes but 'lightweight' will require a 'lightweight' model. Your application for 'domain one' is called open domain question answering (ODQA). Here is a demonstration of ODQA using BERT: ...
Brian O'Donnell's user avatar
1 vote

Fine tuning a Deep Learning model post training

You can use the technique of Transfer Learning to fine-tune your model. You can take the weights from the pre-trained model and then use them as initializations for your own model. Yes!, It is common ...
Faizy's user avatar
  • 1,114
1 vote

can I add to a language model a prompt with output example?

Yes, it is possible to finetune GPT2 to extract relevant data from a given text. For your example, you could add a prompt that looks for a manufacturer, max speed, and horsepower, and then provide an ...
Faizy's user avatar
  • 1,114
1 vote

How to combine pretrained language models with additional feature set?

If you're eventually building a classifier with added features, concatenating the LM output embeddings with those additional features should work, I believe. It seems similar to incorporating non-...
Ashwani Yadav's user avatar

Only top scored, non community-wiki answers of a minimum length are eligible