1
$\begingroup$

Reading Stephen Wolfram's explanation of ChatGPT, it sounds as if first you train a very powerful "autocomplete" function that doesn't know anything specifically about chatbots, and then you create a corpus of chatbot-dialogs on top of that model to show it how to be a chatbot specifically. I'm curious if anyone can explain in a bit more detail how this process works technically, ie taking an already-trained model and "specializing" it with a second training corpus. How are the two neural networks related to one another?

$\endgroup$
2
  • $\begingroup$ It's just more training. There's no "end of training process" that stops you from doing any more training after that; you can just keep doing even more training. There's probably some kind of secret sauce in ChatGPT, but this isn't it $\endgroup$ May 9, 2023 at 12:56
  • $\begingroup$ ChatGPT is a decoder-only transformer model. For a full on (hopefully not too technical explanation), please read this extensive post. It also goes into how the transformer is trained (in two stages). $\endgroup$ May 9, 2023 at 20:09

1 Answer 1

1
$\begingroup$

ChatGPT derives also from InstructGPT that is subject to a reinforcement learning training with human feedback (called RLHF) in order to enble it for instruction following.

Basically, to get a chatGPT you take a GPT model, pre-train it to predict the next token in a sentence (so that it learns the structure of the language), and then RLHF is applied to enable it to follow instructions specified in a prompt. Otherwise it will give you just random (but plausible) complexions of sentences.

The power of chatGPT is the combination of a very large model (billions of parameters), a huge dataset, and a large collection of human annotated prompts and results for the instruction follwing thing.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .