1
$\begingroup$

One of the biggest strengths of ChatGPT is that it generates fitting text with respect to the input query. It usually stays on topic, anwers the question completely and especially does not start talking gibberish or repeating itself.

This behaviour is different when comparing this to older LLMs. For example: GPT2 would usually only stop generating text when it hit the token limit or a predefined stop sequence. Also, it had a much bigger problem with giving repeating answers. Newer models (especially instruction tuned ones) do not suffer from this problems (e.g. llama 2).

So I have 2 questions: What mechanisms/techniques are used in current language models such that...

  1. ...the models know when to stop generating text.
  2. ...the models do not repeat themselfes and stay on topic.

I suspect it might have alot to do with instruction tuning but I am happy to hear from you.

$\endgroup$
1

1 Answer 1

0
$\begingroup$
  1. LLMs have by themself a token used for end of string, so they themself are the one that decide when a generation should stop
  2. This kind of questions are usually pretty opinionated, but the main reasons are bigger models, and more training data

However, tuning the length might also be a bias from the RLHF training some model have received

$\endgroup$
1
  • $\begingroup$ So models are now able to stop by themselves because they can predict the end of string token. But this was already possible with earlier models like GPT2. But you are saying that that one of the reason how they gained the ability to predict that token effectively bigger models with more training data? $\endgroup$
    – Ricu
    Sep 21, 2023 at 17:08

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .