Questions tagged [image-generation]

For questions related to the task of image generation, which can be done, for example, with variational auto-encoders (VAEs) or generative adversarial networks (GANs).

Filter by
Sorted by
Tagged with
12 votes
1 answer
13k views

What are the fundamental differences between VAE and GAN for image generation?

Starting from my own understanding, and scoped to the purpose of image generation, I'm well aware of the major architectural differences: A GAN's generator samples from a relatively low dimensional ...
Alexander Soare's user avatar
10 votes
2 answers
5k views

Using AI to extend an imagine pattern

I have created some nice patterns using the MidJourney tool. I'd like to find a way to extend these patterns, and I was thinking about an AI tool that takes one of these patterns and extends it in all ...
Nicola Lepetit's user avatar
7 votes
1 answer
3k views

How many training data is required for GAN?

I'm beginning to study and implement GAN to generate more datasets. I'll just try to experiment with state-of-the-art GAN models as described here https://paperswithcode.com/sota/image-generation-on-...
gameon67's user avatar
  • 215
6 votes
1 answer
2k views

How does AI 'see' the images it generates- from what perspective?

I've been using AI image generation for a while now, and I've noticed how profoundly AI doesn't seem to see the image as a whole, sometimes generating an image with parts of fingers floating near ...
ben svenssohn's user avatar
6 votes
2 answers
914 views

What is the exact role of model $p_\theta$ in diffusion models for the reverse process?

I'm reading this interesting blog post explaining diffusion probabilistic models and trying to understand the following. In order to compute the reverse process, we need to consider the posterior ...
James Arten's user avatar
6 votes
0 answers
114 views

How can an Artificial Intelligence system be ethically trained to generate art?

There have been a lot of popular AI-generating image systems put out recently, with such systems as Midjourney and Dall-E catching attention with how well put-together many of the automatically ...
Mithical's user avatar
  • 2,905
5 votes
1 answer
1k views

Does MMD-VAE solve the problem of blurred images of vanilla VAEs?

I understand that with vanilla VAEs, there are a few reasons justifying the production of blurred out images. The InfoVAE paper describes the case when the decoder is flexible enough to ignore the ...
Ananda's user avatar
  • 148
5 votes
1 answer
421 views

Context-based gap-fill face posture-mapper GAN

These images are handmade, not auto-generated like they will be in production. Apologies for inaccuracies in the graph overlay. I am trying to build an AI like that displayed in the diagram: when ...
Geza Kerecsenyi's user avatar
4 votes
1 answer
15k views

Why can't AI image generators output verbatim text when prompted to do so?

I want to create a splash screen that includes the name of my project. DALL-E 2 changed some of the letters in the name, even when I tried putting the name of my project in double-quotes (...
Silver Sagely's user avatar
4 votes
1 answer
473 views

What kind of algorithm is used by StackGAN to generate realistic images from text?

What kind of algorithm is used by StackGAN to generate realistic images from text? How does StackGAN work?
Aneesh bhat's user avatar
4 votes
1 answer
779 views

What is the state-of-the-art algorithm for neural style transfer?

I've read the paper A Neural Algorithm of Artistic Style by Gatys et. al. and I find the application of neural style transfer very fun. I also read that Exploring the structure of a real-time, ...
DeepNet's user avatar
  • 41
3 votes
2 answers
1k views

Image-in image-out neural network architectures

With an RGB image of a paper sheet with text, I want to obtain an output image which is cropped and deskewed. Example of input: I have tried non-AI tools (such as ...
logijaz's user avatar
  • 49
3 votes
3 answers
531 views

Is the output of image generation models like Midjourney and Stable Diffusion deterministic?

Assuming the user can set all parameters, including but not limited to the seed. Is the output deterministic? As in, the same set of inputs will create the same image?
Mindwin Remember Monica's user avatar
3 votes
2 answers
2k views

What makes GAN or VAE better at image generation than NN that directly maps inputs to images

Say a simple neural network's input is a collection of tags (encoded in some way), and the output is an image that corresponds to those tags. Say this network consists of some dense layers and some ...
Avetik's user avatar
  • 115
3 votes
1 answer
224 views

Other deep learning image generation techniques besides GANs?

Can you please point me to some resources about image genereation besides GANs? Are there any other techniques throughout history? How did idea of image generation evolved and how it started? I tried ...
i_rezic's user avatar
  • 245
3 votes
3 answers
14k views

Algorithm that creates new images based on other images

Are there any open sourced algorithms that can take a couple of images as an input and generate a new, similar image based on that input? Or are there any resources where I can learn to create such an ...
Mark's user avatar
  • 131
3 votes
1 answer
183 views

beautify an image with reinforcement learning

I am trying to formulate and solve the following problem of image mutation. Suppose I am trying to insert an object image into a "background" image of several objects, and I will need to look for a "...
lllllllllllll's user avatar
3 votes
1 answer
65 views

How do stable diffusion models take the data into account

I'm interested in how text to image models like Midjourney and Dall-E work, where you enter a text prompt and get as output some images. I started reading some papers on it and stumbled upon "...
Rohit Pandey's user avatar
3 votes
1 answer
394 views

How can I generate unique random patterns (similar to the ones in Nutella jars)?

How can I generate unique patterns, as they did for these Nutella jars? See, for example, the video Algorithm designs seven million different jars of Nutella.
Divyansh Gupta's user avatar
3 votes
0 answers
45 views

How random should an untrained generative AI output really be?

I am developing a particular implementation of VAE, and, how usually one does while implementing any architecture, I passed a random input to the model to test if everything worked fine (e.g. check ...
GPU'njoyer's user avatar
3 votes
0 answers
224 views

Reverse Process in Diffusion Model Doesn't Return Original Image

I am attempting to program a Denoising Diffusion Model based on the one introduced in the article by Ho et al. (2020). However, I have run into issues while testing the reverse diffusion process. ...
cabralpinto's user avatar
3 votes
0 answers
49 views

Best Machine Learning Model for "Predicted" Image Generation

I am currently working on undergraduate research to determine hotspots for hand-surface contact. Ideally, I would like to give the model a depth image as input: Example of synthetic depth image and ...
GB-DEV's user avatar
  • 31
3 votes
0 answers
44 views

How do GANs create an image with specific characteristics?

I've seen GANs that do things like convert an image to a painting or this GAN here https://make.girls.moe/#/ that takes in a set of characteristics and generates a waifu with those characteristics. ...
Bryan Tan's user avatar
  • 183
3 votes
0 answers
44 views

How would an AI visualize a story written in natural language?

Can AI transform natural language text describing real scenarios to visual images and videos ? How does as AI interprets say a Harry Potter story if it has to reproduce it in form of videos ? Would be ...
katipra's user avatar
  • 41
2 votes
1 answer
293 views

What does "Gau" in GauGAN stand for?

GauGAN is a neural network architecture from NVIDIA that can create realistic images from semantic maps (and nowadays also textual descriptions).
Moritz Groß's user avatar
2 votes
2 answers
61 views

Can you give me a piece of advise of the network sructure that would be suitable for my task?

I have 2 small images. They are basically the same, but differ in rotation and size. I should estimate the parameters for affine transform to get them similar. What network structure can be suitable ...
Colab Colab's user avatar
2 votes
2 answers
212 views

Is it feasible to use GAN for high-quality image synthesis other than human faces?

The famous Nvidia paper Progressive Growing of GANs for Improved Quality, Stability, and Variation, the GAN can generate hyperrealistic human faces. But, in the very same paper, images of other ...
Bill Yan's user avatar
  • 171
2 votes
4 answers
547 views

Forward Diffusion Process Derivation In Diffusion Models

In papers and other material regarding diffusion models the forward diffusion process is defined by adding a small amount of Gaussian noise to an image $x_0$ for $T$ time steps. In each time step the ...
nkam's user avatar
  • 23
2 votes
1 answer
132 views

Data augmentation for very small image datasets

I am looking for techniques for augmenting very small image datasets. I have a classification problem with 3 classes. Each class consists of 20 different shapes. The shapes are similar between the ...
postnubilaphoebus's user avatar
2 votes
1 answer
74 views

What algorithm would you advise me to use for my task?

I have an image and a mask. I want the image to be the same, but rotated, scaled and positioned like mask. What can I use?
Colab Colab's user avatar
2 votes
1 answer
59 views

How to generate the original image from feature set?

We all know that using CNN, or even simpler functions, like CLD or EHD, we can generate a set of features out of images. Is there any ways or approaches that given a set of features, we can somehow ...
Mary's user avatar
  • 973
2 votes
1 answer
61 views

Creating videos of AI generated photographs

I came across this article today: These faces show how far AI image generation has advanced in just four years. I would never in a million years have guessed that the people on the right (in the first ...
Siddhartha's user avatar
2 votes
0 answers
46 views

Can Inception-ResNet be inverted layer-by-layer?

It has already been shown that by using a normalization layer during training, it is possible to invert a residual network layer-by-layer. I wonder how similar Inception-ResNet is and whether a ...
Richie Bendall's user avatar
2 votes
2 answers
182 views

How does a VGG-based Style-Loss incorporate color information?

I've recently been reading a lot about style transfer, its applications and implications. I understand what the Gram matrix is and does. I can program it. But one thing that has been boggling me is: ...
masterBroesel's user avatar
2 votes
0 answers
692 views

Is there any difference between "image generation" and "image synthesis"?

Generative Adversarial networks (aka GANs) are used for image generation. The phrase image synthesis is also used in literature. I know that the phrase image generation stands for An act of ...
hanugm's user avatar
  • 3,820
2 votes
0 answers
59 views

Suggestion on image inpainting algorithm

Currently, many algorithms are available for image inpainting. In my application, I have some special restriction on training dataset- Let's consider the training dataset of human facial images. ...
user3862410's user avatar
2 votes
0 answers
20 views

How to debug and find neurons that most influenced a pixel in the output image?

I'm building CNN network of Image to Image. After training, I have some bad results in part of the Image. I would like to find the neurons that most influenced those pixels and do retraining only ...
RNN's user avatar
  • 21
1 vote
2 answers
3k views

Are there some known neural networks that, given an input image, can generate a similar image, with the same topic?

Are there some known neural networks that, given an input image, can generate a similar image, with the same topic? Example: input = a photo of a cat on a green table, output = a generated photo of ...
logijaz's user avatar
  • 49
1 vote
1 answer
25 views

Discrepancies in the Exclusion of Elements in Image vs. Text Generation

(The following comments concern DALLE, I have not tested it with other image generating tools, but would be curious to hear if the same happens) When generating images, it seems that ChatGPT (i.e. ...
Léreau's user avatar
  • 113
1 vote
1 answer
727 views

AI generator to restyle a user's photo

I'd like to know what AI-driven image generator (neural network) can be used to take any user selfie and stylize it into, say, a New York Yankees fan. Or as if the user were wearing Gucci clothes. In ...
SharpAffair's user avatar
1 vote
1 answer
142 views

Methods for combining features other than concatenation

I am working image reconstruction project. It is a part of multispectral image fusion. I am referring paper in the link mentioned below. paper link: https://arxiv.org/pdf/2101.09643v1.pdf For image ...
programmer_04_03's user avatar
1 vote
1 answer
430 views

Are there any books that teach text-to-image generation?

I read some of the research papers about text-to-image generation using Imagen, DALL-E 2, etc. but they are heavily scientific and I don't understand a lot of their concepts, so I was wondering are ...
user avatar
1 vote
1 answer
57 views

What is the name of the method for the smart extend of image surroundings?

I'm looking for the name of the method (or algorithms family, or research body) used for the smart extend of image surroundings. For example, the method I'm looking for would take this image: And ...
OfirD's user avatar
  • 113
1 vote
1 answer
239 views

Is there any metric for calculating how natural a single image is given a dataset of the same class images?

Suppose there is a dataset $D$ of images. We have enough number $n$ of images in the dataset and all the images are of a single class. Suppose I generated a new image $I$, which is not present in the ...
hanugm's user avatar
  • 3,820
1 vote
1 answer
243 views

What dataset might Elon Musk's Dall-E have used?

Dall-E, it can generate many imaginative images from the description, even some peculiar images, how did they actually create this kind of dataset to train this AI , because there is not much of that ...
Gavin's user avatar
  • 49
1 vote
0 answers
18 views

sketch guided image generation using cross-attention injection?

I have seen some papers that allow a diffusion model to generate an image that is compatible with a sketch, such as "Sketch-Guided Text-to-Image Diffusion Models" [1], which trains a module ...
Gilad Deutsch's user avatar
1 vote
2 answers
683 views

Do diffusion models take a long time to train?

I am trying to train a diffusion model (from scratch in pytorch). UNet used is not anything too fancy, takes in images and time step as input for about 512 time steps. I am using learnable embeddings ...
Aditya 's user avatar
1 vote
0 answers
64 views

Is AI good at detecting AI-generated content?

Are AI models good at detecting AI-generated image or video content like deep fakes? Which model can we use for the detection of AI-generated image content?
Maciej Łoziński's user avatar
1 vote
1 answer
47 views

Is there a way to reward my Variational-Auto encoder for using less colors while still letting it make creative decisions?

So recently I have been trying to program a Tensorflow and Keras based model that can animate pixel art characters. I use a Variation Auto Encoder with Convolutions, Dense Layers and Upsampling 2d ...
Max's user avatar
  • 13
1 vote
0 answers
66 views

what's the benefit of diffusion models reverse process

To draw a picture, we have two approaches: gradually add(and modify) pixels on white paper. gradually remove(and modify) pixels from a complete random noise picture. Apparently, the diffusion model ...
zzzgoo's user avatar
  • 161