Who invented DAN?

Question

DAN was a prompt that went through many, many iterations during the initial months of ChatGPT’s release to the public. DAN is an acronym which stood for “Do Anything Now”, and was a prompt specifically designed to circumvent the grid lines OpenAI had set to contain ChatGPT’s responses, under its alignment.

DAN stimulated a lot of interaction on related forums like Reddit, where people shared DAN-type output and modifications of the DAN-prompt.

DAN was essentially a first noteable example of prompt injection, in the LLM field, where an attacker can modify the behavior of an LLM system from what its creator intended and programmed it to do.

Out of historical interest, is it known who invented the DAN prompt - which individual or individuals played a role in the evolution of DAN as a cultural phenomenon, and theoretical finding, in the field of prompt engineering?

Answer 1

DAN is invented by a college student Walker, who first posted the prompt in this Reddit post [1].

DAN then became particularly popular. However, there were countless examples of earlier "jailbreaking" prompts on day-1 of ChatGPT. In particular, the "split personality" idea appears before. See, for example, this video, the part from 20:17 by Yannic Kilcher for a summary of various people's Twitter posts.

The idea of the prompt injection attack predates the release of ChatGPT. Riley Goodside posted in this Twitter post a basic "ignored previous instruction" attack against an OpenAI's text-davinci model. Note that this model doesn't have a "safety" filter built-in, so it is not an example of "jailbreak." See also [2].

• [1]: Oremus, Will. "The clever trick that turns ChatGPT into its evil twin". Washington Post (2023).
• [2]: Fábio Perez, Ian Ribeiro. Ignore Previous Prompt: Attack Techniques For Language Models. NeurIPS ML Safety Workshop (2022).