Questions tagged [image-generation]
For questions related to the task of image generation, which can be done, for example, with variational auto-encoders (VAEs) or generative adversarial networks (GANs).
91
questions
12
votes
1
answer
13k
views
What are the fundamental differences between VAE and GAN for image generation?
Starting from my own understanding, and scoped to the purpose of image generation, I'm well aware of the major architectural differences:
A GAN's generator samples from a relatively low dimensional ...
10
votes
2
answers
5k
views
Using AI to extend an imagine pattern
I have created some nice patterns using the MidJourney tool. I'd like to find a way to extend these patterns, and I was thinking about an AI tool that takes one of these patterns and extends it in all ...
7
votes
1
answer
3k
views
How many training data is required for GAN?
I'm beginning to study and implement GAN to generate more datasets. I'll just try to experiment with state-of-the-art GAN models as described here https://paperswithcode.com/sota/image-generation-on-...
6
votes
1
answer
2k
views
How does AI 'see' the images it generates- from what perspective?
I've been using AI image generation for a while now, and I've noticed how profoundly AI doesn't seem to see the image as a whole, sometimes generating an image with parts of fingers floating near ...
6
votes
2
answers
914
views
What is the exact role of model $p_\theta$ in diffusion models for the reverse process?
I'm reading this interesting blog post explaining diffusion probabilistic models and trying to understand the following.
In order to compute the reverse process, we need to consider the posterior ...
6
votes
0
answers
114
views
How can an Artificial Intelligence system be ethically trained to generate art?
There have been a lot of popular AI-generating image systems put out recently, with such systems as Midjourney and Dall-E catching attention with how well put-together many of the automatically ...
5
votes
1
answer
1k
views
Does MMD-VAE solve the problem of blurred images of vanilla VAEs?
I understand that with vanilla VAEs, there are a few reasons justifying the production of blurred out images. The InfoVAE paper describes the case when the decoder is flexible enough to ignore the ...
5
votes
1
answer
421
views
Context-based gap-fill face posture-mapper GAN
These images are handmade, not auto-generated like they will be in production. Apologies for inaccuracies in the graph overlay.
I am trying to build an AI like that displayed in the diagram: when ...
4
votes
1
answer
15k
views
Why can't AI image generators output verbatim text when prompted to do so?
I want to create a splash screen that includes the name of my project. DALL-E 2 changed some of the letters in the name, even when I tried putting the name of my project in double-quotes (...
4
votes
1
answer
473
views
What kind of algorithm is used by StackGAN to generate realistic images from text?
What kind of algorithm is used by StackGAN to generate realistic images from text? How does StackGAN work?
4
votes
1
answer
779
views
What is the state-of-the-art algorithm for neural style transfer?
I've read the paper A Neural Algorithm of Artistic Style by Gatys et. al. and I find the application of neural style transfer very fun.
I also read that Exploring the structure of a real-time, ...
3
votes
2
answers
1k
views
Image-in image-out neural network architectures
With an RGB image of a paper sheet with text, I want to obtain an output image which is cropped and deskewed. Example of input:
I have tried non-AI tools (such as ...
3
votes
3
answers
531
views
Is the output of image generation models like Midjourney and Stable Diffusion deterministic?
Assuming the user can set all parameters, including but not limited to the seed.
Is the output deterministic? As in, the same set of inputs will create the same image?
3
votes
2
answers
2k
views
What makes GAN or VAE better at image generation than NN that directly maps inputs to images
Say a simple neural network's input is a collection of tags (encoded in some way), and the output is an image that corresponds to those tags. Say this network consists of some dense layers and some ...
3
votes
1
answer
224
views
Other deep learning image generation techniques besides GANs?
Can you please point me to some resources about image genereation besides GANs?
Are there any other techniques throughout history?
How did idea of image generation evolved and how it started?
I tried ...
3
votes
3
answers
14k
views
Algorithm that creates new images based on other images
Are there any open sourced algorithms that can take a couple of images as an input and generate a new, similar image based on that input? Or are there any resources where I can learn to create such an ...
3
votes
1
answer
183
views
beautify an image with reinforcement learning
I am trying to formulate and solve the following problem of image mutation. Suppose I am trying to insert an object image into a "background" image of several objects, and I will need to look for a "...
3
votes
1
answer
65
views
How do stable diffusion models take the data into account
I'm interested in how text to image models like Midjourney and Dall-E work, where you enter a text prompt and get as output some images. I started reading some papers on it and stumbled upon "...
3
votes
1
answer
394
views
How can I generate unique random patterns (similar to the ones in Nutella jars)?
How can I generate unique patterns, as they did for these Nutella jars? See, for example, the video Algorithm designs seven million different jars of Nutella.
3
votes
0
answers
45
views
How random should an untrained generative AI output really be?
I am developing a particular implementation of VAE, and, how usually one does while implementing any architecture, I passed a random input to the model to test if everything worked fine (e.g. check ...
3
votes
0
answers
224
views
Reverse Process in Diffusion Model Doesn't Return Original Image
I am attempting to program a Denoising Diffusion Model based on the one introduced in the article by Ho et al. (2020). However, I have run into issues while testing the reverse diffusion process.
...
3
votes
0
answers
49
views
Best Machine Learning Model for "Predicted" Image Generation
I am currently working on undergraduate research to determine hotspots for hand-surface contact. Ideally, I would like to give the model a depth image as input:
Example of synthetic depth image
and ...
3
votes
0
answers
44
views
How do GANs create an image with specific characteristics?
I've seen GANs that do things like convert an image to a painting or this GAN here https://make.girls.moe/#/ that takes in a set of characteristics and generates a waifu with those characteristics.
...
3
votes
0
answers
44
views
How would an AI visualize a story written in natural language?
Can AI transform natural language text describing real scenarios to visual images and videos ? How does as AI interprets say a Harry Potter story if it has to reproduce it in form of videos ? Would be ...
2
votes
1
answer
293
views
What does "Gau" in GauGAN stand for?
GauGAN is a neural network architecture from NVIDIA that can create realistic images from semantic maps (and nowadays also textual descriptions).
2
votes
2
answers
61
views
Can you give me a piece of advise of the network sructure that would be suitable for my task?
I have 2 small images. They are basically the same, but differ in rotation and size. I should estimate the parameters for affine transform to get them similar. What network structure can be suitable ...
2
votes
2
answers
212
views
Is it feasible to use GAN for high-quality image synthesis other than human faces?
The famous Nvidia paper Progressive Growing of GANs for Improved Quality, Stability, and Variation, the GAN can generate hyperrealistic human faces. But, in the very same paper, images of other ...
2
votes
4
answers
547
views
Forward Diffusion Process Derivation In Diffusion Models
In papers and other material regarding diffusion models the forward diffusion process is defined by adding a small amount of Gaussian noise to an image $x_0$ for $T$ time steps. In each time step the ...
2
votes
1
answer
132
views
Data augmentation for very small image datasets
I am looking for techniques for augmenting very small image datasets. I have a classification problem with 3 classes. Each class consists of 20 different shapes. The shapes are similar between the ...
2
votes
1
answer
74
views
What algorithm would you advise me to use for my task?
I have an image and a mask. I want the image to be the same, but rotated, scaled and positioned like mask. What can I use?
2
votes
1
answer
59
views
How to generate the original image from feature set?
We all know that using CNN, or even simpler functions, like CLD or EHD, we can generate a set of features out of images.
Is there any ways or approaches that given a set of features, we can somehow ...
2
votes
1
answer
61
views
Creating videos of AI generated photographs
I came across this article today: These faces show how far AI image generation has advanced in just four years. I would never in a million years have guessed that the people on the right (in the first ...
2
votes
0
answers
46
views
Can Inception-ResNet be inverted layer-by-layer?
It has already been shown that by using a normalization layer during training, it is possible to invert a residual network layer-by-layer.
I wonder how similar Inception-ResNet is and whether a ...
2
votes
2
answers
182
views
How does a VGG-based Style-Loss incorporate color information?
I've recently been reading a lot about style transfer, its applications and implications. I understand what the Gram matrix is and does. I can program it. But one thing that has been boggling me is: ...
2
votes
0
answers
692
views
Is there any difference between "image generation" and "image synthesis"?
Generative Adversarial networks (aka GANs) are used for image generation. The phrase image synthesis is also used in literature.
I know that the phrase image generation stands for
An act of ...
2
votes
0
answers
59
views
Suggestion on image inpainting algorithm
Currently, many algorithms are available for image inpainting. In my application, I have some special restriction on training dataset-
Let's consider the training dataset of human facial images.
...
2
votes
0
answers
20
views
How to debug and find neurons that most influenced a pixel in the output image?
I'm building CNN network of Image to Image.
After training, I have some bad results in part of the Image.
I would like to find the neurons that most influenced those pixels and do retraining only ...
1
vote
2
answers
3k
views
Are there some known neural networks that, given an input image, can generate a similar image, with the same topic?
Are there some known neural networks that, given an input image, can generate a similar image, with the same topic?
Example: input = a photo of a cat on a green table, output = a generated photo of ...
1
vote
1
answer
25
views
Discrepancies in the Exclusion of Elements in Image vs. Text Generation
(The following comments concern DALLE, I have not tested it with other image generating tools, but would be curious to hear if the same happens)
When generating images, it seems that ChatGPT (i.e. ...
1
vote
1
answer
727
views
AI generator to restyle a user's photo
I'd like to know what AI-driven image generator (neural network) can be used to take any user selfie and stylize it into, say, a New York Yankees fan. Or as if the user were wearing Gucci clothes.
In ...
1
vote
1
answer
142
views
Methods for combining features other than concatenation
I am working image reconstruction project. It is a part of multispectral image fusion. I am referring paper in the link mentioned below.
paper link: https://arxiv.org/pdf/2101.09643v1.pdf
For image ...
1
vote
1
answer
430
views
Are there any books that teach text-to-image generation?
I read some of the research papers about text-to-image generation using Imagen, DALL-E 2, etc. but they are heavily scientific and I don't understand a lot of their concepts, so I was wondering are ...
1
vote
1
answer
57
views
What is the name of the method for the smart extend of image surroundings?
I'm looking for the name of the method (or algorithms family, or research body) used for the smart extend of image surroundings.
For example, the method I'm looking for would take this image:
And ...
1
vote
1
answer
239
views
Is there any metric for calculating how natural a single image is given a dataset of the same class images?
Suppose there is a dataset $D$ of images. We have enough number $n$ of images in the dataset and all the images are of a single class.
Suppose I generated a new image $I$, which is not present in the ...
1
vote
1
answer
243
views
What dataset might Elon Musk's Dall-E have used?
Dall-E, it can generate many imaginative images from the description, even some peculiar images, how did they actually create this kind of dataset to train this AI , because there is not much of that ...
1
vote
0
answers
18
views
sketch guided image generation using cross-attention injection?
I have seen some papers that allow a diffusion model to generate an image that is compatible with a sketch, such as "Sketch-Guided Text-to-Image Diffusion Models" [1], which trains a module ...
1
vote
2
answers
683
views
Do diffusion models take a long time to train?
I am trying to train a diffusion model (from scratch in pytorch). UNet used is not anything too fancy, takes in images and time step as input for about 512 time steps. I am using learnable embeddings ...
1
vote
0
answers
64
views
Is AI good at detecting AI-generated content?
Are AI models good at detecting AI-generated image or video content like deep fakes? Which model can we use for the detection of AI-generated image content?
1
vote
1
answer
47
views
Is there a way to reward my Variational-Auto encoder for using less colors while still letting it make creative decisions?
So recently I have been trying to program a Tensorflow and Keras based model that can animate pixel art characters. I use a Variation Auto Encoder with Convolutions, Dense Layers and Upsampling 2d ...
1
vote
0
answers
66
views
what's the benefit of diffusion models reverse process
To draw a picture, we have two approaches:
gradually add(and modify) pixels on white paper.
gradually remove(and modify) pixels from a complete random noise picture.
Apparently, the diffusion model ...