TLDR: It is taught to do that during training
The secret sauce here is not in the architecture but in the fine-tuning part of the training process. The details of which have not been made public yet.
Large-language models (LLMs), such as ChatGPT, indeed do nothing else than continually predict the next token for a given sequence of tokens (the prompt).
LLMs are traditionally simply trained to predict this next token in a self-supervised manner on an immensely large amount of data. However, after having done this, there is a second stage to training, namely fine-tuning. The OpenAI ChatGPT Blog states that it does some human-feedback reinforcement learning.
Human feedback reinforcement learning
In short, human feedback reinforcement learning is nothing else then prompting the model for some input (question), letting it generate several responses, and ordering these answers based on desired output by humans. There are tons of people ranking these answers by hand for fine-tuning these models.
ChatGPT and other Chat LLMs are thus trained to respond to the prompt
this is the reason that
- You prompt seems incomplete
as during fine-tuning this was probably ranked higher by humans than
- stackexchange is awesome.
- the internet is cool.
- the sun comes up in the morning.
In essence, it is thus simply trained to respond that way. It has nothing to do with the architecture, wrappers or anything of the sort.
Now this applies much broader to the general concept of chat. The whole fine-tuning process is basically feeding in questions and receiving answers. So the 'chat functionality' is induced into the model through the training process. However, as you can probably still do, it can finish your stories if you ask it to with prompts such as
You are a writer, finishing stories of others. You start working on "There once was...".
Which is not necessarily a 'chat', but simply next word prediction.