6
$\begingroup$

I know that large language models like GPT-3 are trained simply to continue pieces of text that have been scraped from the web. But how was ChatGPT trained, which, while also having a good understanding of language, is not directly a language model, but a chatbot? Do we know anything about that? I presume that a lot of conversations was needed in order to train it. Did they simply scrape those conversations from the web, and where did they find such conversations in that case?

Improve this question
$\endgroup$
3
  • $\begingroup$ Can you please focus on one model at a time (either LaMDA or ChatGPT), and ask about the other model in a different post? The answers may change from one model to the other, so it's a good idea to focus, otherwise, you may get different answers, for different ways of training the chatbots, which may be what you're looking for, though. $\endgroup$
    – nbro
    Commented Dec 29, 2022 at 10:05
  • $\begingroup$ @nbro While I would really be interested in knowing about all promising approaches, I chose to focus on ChatGPT since that's the only chatbot that I've personally tried (in recent times at least). $\endgroup$
    – HelloGoodbye
    Commented Jan 2, 2023 at 1:30
  • 2
    $\begingroup$ This article describes it well: "ChatGPT: How Does It Work Internally?" $\endgroup$
    – Snehal Patel
    Commented Jan 2, 2023 at 3:56

1 Answer 1

Reset to default
6
$\begingroup$

The key ingredient is called Reinforcement Learning from Human Feedback (RLHF), that is having humans rate the model answers and use the feedback to guide the model training.

enter image description here

The official blog explains this fairly well.

Improve this answer
$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .