I'll preface this by saying that I have little experience in artificial intelligence, so this might be a naive question.
However, in light of the recent controversy surrounding ChatGPT's inability to say "I don't know" and its tendency to instead make things up, I couldn't help but wonder:
why not simply train a deep learning algorithm, even as simple as a large ANN, on all the data that ChatGPT was trained on plus a collection of ChatGPT responses manually labelled as accurate or inaccurate?
In fact, one might even imagine a GAN system, with one NN taking a ChatGPT response as input and an improved response/changes to response as output, and the other assessing the veracity of the improved response.
Compared to what ChatGPT is already capable of, to a layman like me, this looks like a trivial task - making sure the input is consistent with the right portion of the training data, or with some comparatively simple patterns within said data, seems infinitely shorter of a task than abstract or original thinking.
So why was such a system not implemented? It's just about the most glaring solution to this problem possible, so there must be something wrong with it if OpenAI still haven't implemented it. Which begs the question: where does it fall apart?
I tried looking for an answer to this question online, but haven't found anything.