One of the main concerns of using ChatGPT answers on Stack Exchange is that it may copy verbatim or almost verbatim some text from its training set, which may infringe the source text's license. This makes me wonder how much of the ChatGPT output is copied from its training set (vs. being abstractively generated).
$\begingroup$
$\endgroup$
7
asked Dec 21, 2022 at 11:52
-
1$\begingroup$ Is there any evidence for this "it may copy verbatim or almost verbatim some text from its training set"? This may be true, but I am wondering if there is any evidence. I never tried it, so I don't know. $\endgroup$– nbroCommented Dec 21, 2022 at 12:55
-
1$\begingroup$ I think this is incorrect. The real problem is that it's generating text based on the probability that a certain token appears in a given position given all of the other tokens so far... which means that the result might be interesting, but isn't necessarily anything like accurate. $\endgroup$– David HoelzerCommented Dec 21, 2022 at 13:23
-
$\begingroup$ @nbro I don't know any. I haven't tried much. $\endgroup$– Franck DernoncourtCommented Dec 21, 2022 at 14:04
-
$\begingroup$ @DavidHoelzer that's indeed another concern. meta.stackexchange.com/q/384410/178179 ; meta.stackexchange.com/q/384652/178179 $\endgroup$– Franck DernoncourtCommented Dec 21, 2022 at 14:04
-
$\begingroup$ This is impossible to answer since the training set is not public. $\endgroup$– Dr. SnoopyCommented Dec 22, 2022 at 10:25
|
Show 2 more comments
1 Answer
$\begingroup$
$\endgroup$
From the paper Language Models are Changing AI: The Need for Holistic Evaluation (Authors: Rishi Bommasani and Percy Liang and Tony Lee; Website):
Memorization of copyrighted/licensed material. We find that the likelihood of direct regurgitation of long copyrighted sequences is somewhat uncommon, but it does become noticeable when looking at popular books. However, we do find the regurgitation risk clearly correlates with model accuracy: InstructGPT davinci v2 (175B*), GPT-3 davinci v1 (175B), and Anthropic-LM v4-s3 (52B) demonstrate the highest amount of verbatim regurgitation in line with their high accuracies.
[...]
To further explore the results for this targeted evaluation, see https://crfm.stanford.edu/helm/v1.0/?group=copyright_text , https://crfm.stanford.edu/helm/v1.0/?group=copyright_code and Figure 39. We evaluated various models for their ability to reproduce copyrighted text or licensed code. When evaluating source code regurgitation, we only extract from models specialized to code (Codex davinci v2 and Codex cushman v1). When evaluating text regurgitation, we extract from all models except those specialized to code. Overall, we find that models only regurgitate infrequently, with most models not regurgitating at all under our evaluation setup. However, in the rare occasion where models regurgitate, large spans of verbatim content are reproduced.
ChatGPT shouldn't be too far away from InstructGPT davinci v2.
answered Mar 14, 2023 at 1:06