Is the GPT-4 for text the same model that can input and output images?

Question

Currently, the published GPT-4 can input and output text. A version of GPT-4 that can input and output text and images exists, according to the technical report, but is not yet publicly available.

I suspect that they are the same. Only the interface for image input and output is not publicly available.

That is because GPT-4 shows some understanding of images if you find a way to output them. One way is asking it to write a program that produces the image output. And that actually works, demonstrated below:

Prompt: "Write a program that creates a picture of HAL 9000"
[System prompt empty]

The result is this image (created by following the directions in the text output):

The following is the unedited GPT-4 output (formatting in the original):

---

Creating a visual program requires a programming language with graphics libraries or frameworks, as well as a specific platform or environment to run the code. Here's an example using Python and the Pygame library to create a simple picture of HAL 9000:

1. First, you need to install the Pygame library if you haven't already. You can do this by running the following command in your terminal:

pip install pygame

2. Next, create a new Python file (e.g. hal9000.py) and paste the following code:

import pygame

# Initialize Pygame
pygame.init()

# Set up the display
width, height = 400, 400
screen = pygame.display.set_mode((width, height))
pygame.display.set_caption("HAL 9000")

# Define colors
BLACK = (0, 0, 0)
RED = (255, 0, 0)
WHITE = (255, 255, 255)

# Draw HAL 9000
def draw_hal9000(surface):
    surface.fill(BLACK)
    
    # Draw the outer circle
    pygame.draw.circle(surface, WHITE, (width // 2, height // 2), 100)
    
    # Draw the inner circle
    pygame.draw.circle(surface, RED, (width // 2, height // 2), 80)

    # Draw the lens
    pygame.draw.ellipse(surface, BLACK, (width // 2 - 40, height // 2 - 60, 80, 120))

    # Draw the lens reflection
    pygame.draw.ellipse(surface, WHITE, (width // 2 - 20, height // 2 - 45, 15, 25), 2)

# Main loop
running = True
while running:
    for event in pygame.event.get():
        if event.type == pygame.QUIT:
            running = False
    
    draw_hal9000(screen)
    pygame.display.flip()

# Clean up
pygame.quit()

3. Save the file and run the program using the following command in your terminal:

python hal9000.py

This code will create a simple picture of HAL 9000 using the Pygame library. The main loop will keep the program running until you close the window.

Note: This example assumes that you have Python and pip (package installer for Python) already installed on your system. If you don't have Python installed, you can download it from the official website: https://www.python.org/downloads/

---

Answer 1

No chatGPT is a text-model only.

If you know that HAL 9000 is mainly a red circle, and you know how to generate a red circle in python, you don't need to visually see the image to provide the code.

There are many examples where the model can generate very impressive instructions, for example how to make 3d with blender. But that does not mean ChatGPT literally opens the program and works with it, before providing instructions.

It is discussed here by OpenAI the use of GPT technology as an image-model.

Answer 2

They are not the same model, and the text-only model is definitively able to output graphical content via various forms of code; see e.g. Sparks of Artificial General Intelligence: Early experiments with GPT-4 for a source on this and for some concrete examples.