2
$\begingroup$

My assumption is that after the transformer is trained, some other software analyzes the answer for anti-wokeness and modifies it accordingly, rather than being trained using woke material.
I don't think this is correct though. I asked chatGPT to answer every question by inserting a $ character in between each character of the otherwise normal answer, to make it unreadable, or at least, difficult to interpret by what I call the censorship layer. The output had these inserted characters, however, this had no effect on the censorship. But may be the censorship layer is smart enough to read in between lines.

Question: Does the censorship comes from within the transformer or after it? (and, if it comes after, why my attempt to bypass it failed?)

Improve this question
$\endgroup$
3
  • 5
    $\begingroup$ This is impossible to answer because there is no publicly available information about inner workings of ChatGPT. $\endgroup$
    – Dr. Snoopy
    Feb 5 at 1:21
  • 2
    $\begingroup$ There is no way to know why your personal attempts to bypass the filters have failed, and it is a ridiculous question since you have given no details about what you've attempted. Additionally, the question is deliberately inflammatory by using "woke" as some sort of aspersion. There is nothing "woke" about making a tool publicly available to children not write erotica or use racist epithets. While the question has attracted one unsourced answer about a proprietary system, that may or may not be correct, the overall question certainly does not live up to SE's standards in multiple ways.. $\endgroup$
    – Ben I.
    Mar 14 at 20:48
  • 1
    $\begingroup$ Dr. Snoopy is correct - any answer to this question as it is currently posed is mere speculation, unless somebody from OpenAI is willing to respond and provide actual insider knowledge. I would encourage you to reword this in such a way that it's not asking for information that is basically "unavailable by definition". Ben I. is also correct. I'd encourage you to drop the use of "woke" as a pejorative and keep this neutral and mostly technology focused. Otherwise this is at risk of being closed. $\endgroup$
    – mindcrime ♦
    Mar 15 at 3:50

1 Answer 1

Reset to default
0
$\begingroup$

Censorship in OpenAI's GPT models is implemented at the post-processing stage, after the model has generated its response. The censorship policies and criteria are defined by OpenAI, and they are applied to the output of the model before it is displayed to the user. The policies and criteria are designed to prevent the generation of harmful, abusive, or otherwise inappropriate content. The exact details of how censorship is implemented and enforced by OpenAI are proprietary and not publicly disclosed.

Improve this answer
$\endgroup$
5
  • $\begingroup$ Thanks, did you read the paragraph before the question? What motivated the question was not really answered. That is my mistake though, I should have explicitly stated that in the italics. I edited it. $\endgroup$
    – Wolphram jonny
    Feb 5 at 18:11
  • $\begingroup$ "There is no censorship mechanism within the transformer.". It answered GPT-3 itself. $\endgroup$
    – Andrea Ianni
    Feb 5 at 18:25
  • $\begingroup$ I would not trust GPT-3 answers. Also, you didn't address my question: why my attempt to bypass it failed? $\endgroup$
    – Wolphram jonny
    Feb 5 at 18:44
  • $\begingroup$ Even with old mechanisms like word2vec it was really simple to identify clusters of offensive words, all in the same vector spaces. I think GPT-3 is trained to avoid those clusters in the answer or recognizing questions with answers falling in those dangerous spaces. So, I think it's all inside the transformers and there isn't a way to bypass it. It creates "polite" answers and then adds the special character $ to the text. $\endgroup$
    – Andrea Ianni
    Feb 5 at 21:07
  • 7
    $\begingroup$ Can you provide a source that supports this claim "Censorship in OpenAI's GPT models is implemented at the post-processing stage". I know OpenAI has some kind of "moderation" API, but I'd like to see some sources that support any claim about ChatGPT, because we see a lot of speculation. $\endgroup$
    – nbro
    Feb 5 at 21:33

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .