Discrepancies in the Exclusion of Elements in Image vs. Text Generation

Question

(The following comments concern DALLE, I have not tested it with other image generating tools, but would be curious to hear if the same happens)

When generating images, it seems that ChatGPT (i.e. DALLE) fails to exclude specific elements from an image. As an example, here is the image I got based on the simple prompt:

Generate a picture of a zoo, but make sure that no elephants are in the picture.

Here is the prompt that ChatGPT generated and used for image generation: "A bustling zoo scene filled with a variety of animals such as giraffes, zebras, lions, and monkeys. The zoo is lively with visitors walking around, looking at different animal exhibits. There are trees, benches, and pathways that lead to different sections of the zoo. Food stands and souvenir shops are scattered throughout, adding to the busy atmosphere. The sky is clear and sunny, perfect for a day out. Make sure there are no elephants in the picture".

When it comes to text generation, this does not seem to be an issue. Using the prompt

Write a small poem about a zoo, but make sure that no elephants are mentioned.

I have never gotten a result that mentioned elephants. If the comment concerning elephants is left out however, most poems will talk about animals in the zoo and then also mention them.

Given the observed challenges in the exclusion of elements (like elephants) in pictures, but the relative ease of that for texts, what are the fundamental reasons for this? Is this due to some inherent differences in training or model architecture in both applications?

Answer 1

The difference is due to qualities of text training data used in training the two types of model.

The LLM models are trained on large amounts of text, and learn grammar rules very well. There are lots of examples of negation in the training data. The trained models can process what negation means in phrases, and what that may mean for the following phrases, i.e. what they are likely to generate when a term such as "no elephants" is in the previous context.

The image models are trained on images paired with image descriptions. The text is generally simpler, and there is less of it overall than with LLM training data (hundreds of millions of sentences instead of hundreds of billions). Part of the image training process involves creating a shared multimodal "concept space" where text and image descriptions both resolve to the same descriptive vector*. This allows the generator to measure how far away a generation in progress is from an image with the same description as its text input. It can then use that to adjust the image pixels towards something with a more similar concept vector to the text.

The thing that is generally missing from the image/text pairs is negation. It is rare to have images described by what is not in them, and definitely not text generally covering all the details in a complex image - like your zoo image - that are not there. There are therefore very few places in the "concept space" which cover the negation of some element or other. In fact you often get the opposite effect in a generator like DALL.E - the more you mention an element, positively or negatively, the more prominently it will appear in the image.

This issue is common across image generators. In Google's blog about their model Parti, they give an example of this as one of the failure modes.

Image generators can and do allow for this. They allow the use of negative prompts (Stable Diffusion), or prompt weights which can be negative (Midjourney). These work by adding the negation of the concept vector, based on the text in the separate negative prompt. The image, whilst generating, will be adjusted to not be like, or not contain, the concept in the negative prompt.

I don't know why DALL.E does not support the use of negative prompts.

---

* It's more complex than that, what I describe is very roughly how CLIP guidance works.