1
$\begingroup$

I'll preface this by saying that I have little experience in artificial intelligence, so this might be a naive question.

However, in light of the recent controversy surrounding ChatGPT's inability to say "I don't know" and its tendency to instead make things up, I couldn't help but wonder:

why not simply train a deep learning algorithm, even as simple as a large ANN, on all the data that ChatGPT was trained on plus a collection of ChatGPT responses manually labelled as accurate or inaccurate?

In fact, one might even imagine a GAN system, with one NN taking a ChatGPT response as input and an improved response/changes to response as output, and the other assessing the veracity of the improved response.

Compared to what ChatGPT is already capable of, to a layman like me, this looks like a trivial task - making sure the input is consistent with the right portion of the training data, or with some comparatively simple patterns within said data, seems infinitely shorter of a task than abstract or original thinking.

So why was such a system not implemented? It's just about the most glaring solution to this problem possible, so there must be something wrong with it if OpenAI still haven't implemented it. Which begs the question: where does it fall apart?

I tried looking for an answer to this question online, but haven't found anything.

Improve this question
$\endgroup$
6
  • 3
    $\begingroup$ Have you considered how much data is required to ascertain if a response is correct or not? Or that the task is ambiguous (defining correct and incorrect is not easy, humans could disagree)? Large language models are trained on billions of data points, it is not feasible to label the same quantity of data points for correctness. $\endgroup$
    – Dr. Snoopy
    Commented Dec 7, 2022 at 16:30
  • $\begingroup$ @Dr.Snoopy I thought the data that GPT was trained on would be sufficient - ChatGPT seems to have all the necessary information (at least that has been my experience), just not its accuracy. As to the data points, I'm sure it can't be too hard to sample statements from encyclopaedias to generate both correct statements and incorrect statements (just negate the correct statements). Otherwise, even UGC could probably work given ChatGPT was trained UGC conversations. $\endgroup$
    – Max
    Commented Dec 7, 2022 at 23:09
  • 1
    $\begingroup$ You need labels about correct/incorrect, this is not part of the train set of ChatGPT, who would make those labels? And what about missing knowledge? Encyclopedias do not cover everything, and do not cover possible wrong things. $\endgroup$
    – Dr. Snoopy
    Commented Dec 7, 2022 at 23:10
  • $\begingroup$ Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer. $\endgroup$
    – Community Bot
    Commented Dec 7, 2022 at 23:20
  • $\begingroup$ @Dr.Snoopy My proposal was that you'd have a compressed version of that data (e.g. just giving the model the ability to look things up in the database using simple keys, with the database being classified maybe even by ChatGPT itself) combined with a sample statement as a single data point, and the correct/incorrect labels for those data points would be determined based on whether the statement is a restatement, or a contradiction, of the sampled encyclopedia statement. As to missing knowledge, I expect the discriminator to have less confidence in classifying missing knowledge. $\endgroup$
    – Max
    Commented Dec 7, 2022 at 23:25

2 Answers 2

Reset to default
9
$\begingroup$

You are massively underestimating the difficulty of the task, you would need:

  • A dataset containing labels of correct/incorrect, at a similar scale (billions of data points).
  • A definition of correct/incorrect, which by itself is difficult, just think that some people believe anything that does not fit their world view to be fake news or lies.

Then consider, who would label this dataset? I don't think there is a train set containing this kind of data. You would have to gather text and have a human label it, at billion scale, would take a lot of time and effort.

More importantly, there could be controversial topics where there is no a clear definition of right or wrong. What is the label in this case? Also there is a huge class imbalance, you can have some data points for correct labels, but there are infinite ways to be incorrect. So any dataset you have would be biased towards the correct class.

The point of machine learning is generalization, I don't think you can just grab some random data and generalize this idea to absolute correct/incorrect. Even doing this for neural networks with images is very difficult.

And also generalization, you should consider that even if you somehow train a classifier to output correct/incorrect, these predictions themselves could be incorrect (outputting correct when it is actually incorrect and viceversa), so you do not solve any problem really.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Moreover, in many cases, what is correct in one context is not in another but this nuanced knowledge is simple to lose track of. Is XYZ the greatest player in the world is a fact that depends on the time period in question. The answer in 1923 is different from the answer in 2023, and maybe different in 2024, independent of how imprecise or subjective the question is. $\endgroup$
    – Anirban Mukherjee
    Commented Jun 16, 2023 at 23:01
0
$\begingroup$

Correcting the output of one NN with a second NN is not a very good approach. If you have extra data which is used to train the second NN exclusively, why not use this data to train the original NN in the first place?

And if there is no extra data available, then it's unlikely that the second NN will be effective at correcting the first one. The original NN makes mistakes when it needs to extrapolate too much in the absence of a good match with the training data. The second NN will run into the same extrapolation incertitude again.

Improve this answer
$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .