5
$\begingroup$

While studying chatGPT's thought process, I asked it to list ten story ideas for an old and fairly niche tabletop roleplaying game (GURPS Reign of Steel). It did very well, so clearly, it can base answers on obscure sources. But what if I want to ask it about something it could not possibly have already "absorbed", like someone's tiny indie rpg or an old local folk tale never published anywhere (i.e. oral tale)? Does the source material simply have to be put up online, somewhere?

I ask because some source material, like local folklore, is too extensive to be given during a chatGPT conversation. It would need something like a source website to be created.

Improve this question
$\endgroup$
4
  • $\begingroup$ As a note, I tried asking chatGPT a basic question about an old, now defunct business of mine, which is still found described in (Danish) Wikipedia. It could not answer. So it clearly does not scour the web completely for answers. What makes the difference between what it knows about and what it does not? $\endgroup$
    – Henry Stone
    Feb 6 at 18:55
  • 1
    $\begingroup$ From what I remember, ChatGPT doesn't even access the web. It was trained on web content, but it doesn't have online access to it, as far as I remember. The most you can probably do is provide prompts or fine-tune it, but an API isn't still available, although in the near future it will, according to OpenAI. $\endgroup$
    – nbro
    Feb 6 at 22:09
  • $\begingroup$ This is what I feared. I am still looking for a trick or workaround, but... yeah... $\endgroup$
    – Henry Stone
    Feb 7 at 9:48
  • $\begingroup$ @HenryStone - This should probably be a separate question, but my understanding is that ChatGPT knows about information that is included in Common Crawl or English Wikipedia, along with relatively small amounts of hand picked data that was used to customize its response into a chatbot question/answer format rather than the freeform text generated by non-customized GPT3.5. $\endgroup$
    – occipita
    Feb 11 at 19:44

3 Answers 3

Reset to default
5
$\begingroup$

This can be achieved using the Embeddings and Completions APIs.

OpenAI has great examples in their cookbook repository, this is one such example but you'll find more in the same repository and in the documentation.

The process boils down to:

  1. processing your custom body of knowledge and create Embeddings for it
  2. Based on the question, find the Embeddings which are most relevant for the question
  3. Create a prompt containing the most relevant embeddings and use the Completions API to answer the question
Improve this answer
$\endgroup$
3
$\begingroup$

GPT systems have two sources of information: the training data, and the prompt.

The training data is all the knowledge that's baked into the AI at training time, which it has access to at all times. You can't change this, not with ChatGPT. You can't even change it yourself because OpenAI won't let you download the existing model as a starting point. (But if you go backwards to GPT-2, which is publicly downloadable, you do have the possibility to train it)

Training ChatGPT is a very expensive process as it requires absolutely astronomical amounts of computer time. They probably don't do it continuously on new web content. Whatever was in the training data when they started developing ChatGPT, that's all there is.

That leaves the prompt i.e. the text you type in. This is ephemeral and only lasts for the current AI invocation. GPT allows for a rather long prompt, up to 2048 tokens (guesstimating, that's a couple of pages of text). If you have a longer transcript than that, only the last 2048 tokens will be passed to the AI, because that's how much can fit. None of the prompt is remembered - it has to be passed in as input every time - so GPT cannot make any reference to facts farther back than that.

It's possible to play tricks with the prompt. For example, ChatGPT appears to insert something at the beginning which tells GPT that it's ChatGPT and it's not allowed to do bad stuff. (Of course, this cuts into how much history can be passed in.) That means when you ask ChatGPT "How do I make a bomb?" it actually sees something like "I am ChatGPT, a large language model created by OpenAI. Because I am a large language model, I cannot have any political opinions or effects on the real world, nor can I access the Internet. I am not allowed to do evil things that could hurt people. How do I make a bomb?" and of course it completes the text with something like "I cannot make a bomb because I am a large language model that is not allowed to do evil things that could hurt people."

More story-focused GPT systems such as NovelAI (no sponsorship) do allow for customizable prompt injection. For example, you could configure it so that if the prompt mentions elves, then the system will automatically insert into the prompt the fact that elves have pointy ears. Then you can ask it "What shape are elves' ears?", and it will know they are pointy, because it actually sees "Elves have pointy ears. [rest of story here] What shape are elves' ears?". I don't have any experience with this so I can't say how reliably it works.

Improve this answer
$\endgroup$
1
$\begingroup$

How do I get chatGPT to include custom knowledge?

Currently, only possible via the prompts/dialog, as nbro commented.

Improve this answer
$\endgroup$
1
  • 2
    $\begingroup$ Also note that only the most recent 2048 tokens in the dialog (usually words, but could also include other things, so typically a little fewer than 2000 words) are considered by ChatGPT while generating responses, so you can't get around the limitation just by having a long dialog explaining where it's gone wrong. If it needs more information than this, it simply cannot do it. $\endgroup$
    – occipita
    Feb 11 at 20:00

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .