2
$\begingroup$

All the prompt engineering techniques I've seen seem to focus on telling the model what to do e.g. Few-Shot Prompting.

Is there any value in giving the model examples of what not to do? Can you link me to any papers/techniques on the topic?

Example

I am building a bot to improve students' foreign language writing skills.

Bad output: Corrected spelling of 'heisse' to 'heiße' because 'heiße' is the correct spelling in German.

Better output: Corrected spelling of 'heisse' to 'heiße' because 'ss' can be combined to form 'ß' in German. I could solve this specific problem using few-shot prompting. But really, I want to tell the model "don't give answers like 'this is how it is done in German', instead explain what is being done and the reasons for it".

I may have answered my own question there... just put it what I said above in the system prompt?

$\endgroup$
4
  • $\begingroup$ I think the question in the title is a bit vague and not very interesting. You can certainly create prompts where you ask the model not to do something. The more interesting question is - how effective are they and is there any research on the topic? So, I'd reformulate the title to be more specific. From my experience, if we ask the model not to be verbose, it can still be verbose. But this doesn't say much because if you ask the model to do something it might also not do it. $\endgroup$
    – nbro
    Commented Jul 13, 2023 at 0:29
  • $\begingroup$ You can fine-tune davinci-003 for your case. With enough examples, it should start outputting the required information. $\endgroup$
    – Chinmay
    Commented Jul 13, 2023 at 3:40
  • $\begingroup$ @nbro yes the idea of them being effective/whether research exists was the idea $\endgroup$
    – codeananda
    Commented Jul 13, 2023 at 11:23
  • $\begingroup$ @Chinmay so you're saying that yes you can do negative prompting and that it is effective? $\endgroup$
    – codeananda
    Commented Jul 13, 2023 at 11:23

3 Answers 3

3
$\begingroup$

The Super-NaturalInstructions paper tested including negative (i.e., incorrect examples) during testing and instruction-tuning, along with adding explanations for why an included few-shot example was correct or incorrect. It seems like it doesn't improve accuracy, at least in their configuration (see Table 4).

Although, if you're using a model like ChatGPT, as @nbro mentioned, you could probably just prompt the model to not do something.

$\endgroup$
3
$\begingroup$

Actually, my previous answer is a bit out of date. There's a recent paper that does what you're looking for: "PREADD: Prefix-Adaptive Decoding for Controlled Text Generation."

During inference-time, they do two forward-passes. The first is for your normal generation, and the second is prefixed with a prompt that causes undesirable outputs.

In your example, your normal forward pass could be: "Improve the student's grammar." and the bad forward pass could be "You are a terrible language teacher. Improve the student's grammar." You can then sample tokens that are unlikely under this second (bad) forward-pass, but likely in the first forward-pass. See Section 3 in the paper for the exact formulation.

$\endgroup$
2
$\begingroup$

Does Negative Prompting Exist?

Yes, e.g. in some text-to-image generation models such as https://app.leonardo.ai/ai-generations:

enter image description here

One can run such negative prompts on one's computer, e.g. with Automatic1111's Stable Diffusion WebUI (Windows 10/11 installer):

enter image description here

The paper {1} describes one example of how negative prompts can be implemented in LLMs.


References:

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .