5

The Focus of This Question "How can ... we process the data from the true distribution and the data from the generative model in the same iteration? Analyzing the Foundational Publication In the referenced page, Understanding Generative Adversarial Networks (2017), doctoral candidate Daniel Sieta correctly references Generative Adversarial Networks, ...


4

Compare generated and real data All the results produced by G are always considered "wrong" by definition, even for a very good generator. You provide the discriminative neural network $D$ with a mix of results generated by the generator network $G$ and real results from an outside source, and then you train it to distinguish if the result was produced by ...


3

Not necessarily it depends on the function of the problem space for both the GANs. A real world example: a batter's reaction time and a pitchers max speed are actual bounded values based on genetics and physics. If the max speed a pitcher can pitch is greater than the max reaction time a human needs to effectively hit against them they will permanently be ...


3

With Generative Adversarial Networks, all the generator cares about is fooling the discriminator. There's no requirement to be clever, or exhaustive, or make efficient use of the input space. As long as the discriminator returns "real" (vs. "fake") the generator "wins". The hope is that as the generator and discriminator are trained simultaneously, each ...


2

A discriminative network ($D$) learns to discriminate by definition - we provide it with the true and the generated data, and let it learn by itself how to discriminate between the two. Therefore, we expect network $D$ to improve the ability of network $G$ to generate better and better images (or other kind of data), as it try to "trick" network $D$ by ...


2

Let's start at the beginning. GANs are models that can learn to create data that is similar to the data that we give them. When training a generative model other than a GAN, the easiest loss function to come up with is probably the Mean Squared Error (MSE). Kindly allow me to give you an example (Trickot L 2017): Now suppose you want to generate cats ; ...


2

in my experience GANs work really well for the scenario of semisupervised learning, where you don't necessarily have labels for all your class B data but you do have a balanced dataset. In my (limited) experience, you do have to have a balanced (in numbers) set of A and B objects, even though you are not sure of the labels. And yes, GANs can overfit to ...


2

I think you'll enjoy this work from Apple on improving the realism of synthetic images. Essentially what you need to do is generate a synthetic image then have your GAN modify the synthetic image so that a 1) a discriminator thinks it is real while also 2) not changing the gross structure of the image very much (so the traffic sign doesn't move) - yes, this ...


2

The terms Supervised Learning and Unsupervised Learning predate the invention of the application of artificial networks to a generative and discriminative network pair, which was the first popular generative topology. The existence of labeling is the key distinction between the two. Even partial labeling indicates supervision, as odd as that jargon is, ...


2

The key is: VAE usually use a small latent dimension, the information of input is so hard to pass through this bottleneck, meanwhile it tries to minimize the loss with the batch of input data, you should know the result -- VAE can only have a mean and blurry output. If you increase the bandwidth of the bottleneck, i.e. the size of latent vector, VAE can get ...


2

In essence, Variational Autoencoders learn an "explicit" distribution of the data by trying to fit the data via a multi-dimensional Gaussian/Normal distribution. However, Generative Adversarial Networks learn an "implicit" distribution of data meaning that you cannot directly sample them. Also, due to the deterministic nature of neural networks GANs tend to ...


2

Latent is a synonym for hidden. Why is it called a hidden (or latent) variable? For example, suppose that you observe the behaviour of a person or animal. You can only observe the behaviour. You cannot observe the internal state (e.g. the mood) of this person or animal. The mood is a hidden variable because it cannot be observed directly (but only ...


2

It is called a Latent variable because you cannot access it during train time (which means manipulate it), In a normal Feed Forward NN you cannot manipulate the values output by hidden layers. Similarly the case here. The term originally came from RBM's (they used term hidden variables). The interpretation of hidden variables in the context of RBM was that ...


2

The main reason that the discriminator is trained concurrently with the generator is to provide (at least in theory) a smooth and gradual learning signal for the generator. If we trained the discriminator on only the input data, then, assuming our training algorithm converges well, it should quickly converge to a fixed model. The generator can then learn to ...


2

To understand this equation first you need to understand the context in which it is first introduced. We have two neural networks (i.e. $D$ and $G$) that are playing a minimax game. This means that they have competing goals. Let's look at each one separately: Generator Before we start, you should note that throughout the whole paper the notion of the data-...


2

I'll answer your questions one by one: In this equation are the $E_{z \sim p_z(z)}$ and $E_{x \sim p_{data}(x)}$ the means of the distributions of the mini batch samples? So let's take the first part $E_{x \sim p_{data}(x)}[log \,D(x)]$. This is read as the "expected value of $log \, D(x)$, where $x$ is sampled from $p_{data}(x)$". So, in simpler terms ...


2

To put it simply GANs suffer from a problem of uneven learning rate. Imagine the learning rate of a pitcher and hitter if the pitcher gets to a point where they can throw much better than the hitter can hit then the hitter may fall into a 'training pit' as to be unable to ever learn how to hit from the pitcher. This follows a continues relationship in ...


2

Why AI is (or not) a good option for the generation of random numbers? AI approaches are generally not good for generating random numbers, for these reasons: Similar to why they are not good for adding numbers, there already exist many strong pseudo-random and "true" random sources, possible without using any AI approach, and demonstrably good enough for ...


1

I don't think he said that at all. Going back to the talk you'll see he mentions mode collapse comes from the naivete of using alternating gradient-based optimization steps because then $min_{\phi}max_{\theta}L(G_\phi, D_\theta)$ starts to look a lot like $max_{\theta}min_{\phi}L(G_\phi, D_\theta)$. This is problematic because in the latter case the ...


1

Lets start with question 1) how does JS-divergence handles zeros? by definition: \begin{align} D_{JS}(p||q) &= \frac{1}{2}[D_{KL}(p||\frac{p+q}{2}) + D_{KL}(q||\frac{p+q}{2})] \\ &= \frac{1}{2}\sum_{x\in\Omega} [p(x)log(\frac{2 p(x)}{p(x)+q(x)}) + q(x)log(\frac{2 q(x)}{p(x)+q(x)})] \end{align} Where $\Omega$ is the union of the domains of $p$ and ...


1

Your mistake is that you think that the referenced $V(D,G)$ is the deifinition of the cross entropy! Indeed, the cross entropy is defined base on the negative value of the $V(D,G)$. Hence, if you consider the minus behind the $V(D,G)$ ($-V(D,G)$) the sentence will be meaningful.


1

The model (that I know of) which most resembles your description is the auto-encoder, which is trained to learn a compact representation (a vector) of the input, which can later be used to reconstruct the original input. In a certain way, this compact representation (implicitly) encodes the most important features of the input. In particular, you may be ...


1

Your goal is to model a distribution when constructing a GAN, therefore you need a way to be able to sample that distribution. The noise's purpose is so you can do this. Generally, it's drawn from a distribution that is computationally easy to draw from (like a gaussian). You are modeling the generator $G(X)$ where $X \sim N(\mu, \sigma^2)$. this means $...


1

It's not supposed to be derived from some equation. That is the basic premise under which GANs work. The output of the Generator $G(z)$ is fed as an input $x_g$ to the discriminator.


1

As @Clement mentions, text_gen_description gives a good overview!, but the paper seqGAN paper describes the REINFORCE approach more in depth, as they are the first to do it (i believe). This is probably the approach most take now of days when going the GAN route. Note that just basic MLE training has shown promise with openAI's GPT2. When i need a text ...


1

With a Google Cloud V100 GPU the GAN would run a week to two with default parameters. Does this sound realistic time for this kind of dataset? It's definitely not feasible for me. Yes, V100s are quite beefy. You shouldn't even need a week. Obviously this is based on my experience with various problems, rather than a concrete calculation. Is 4000 ...


1

Let me first provide a brief introduction first GANs as VAEs are generative models which means they learn exactly what you described: to map a typically small dimensional vector/tensor into a higher dimensional one, which in your case is (interpreted as) an image. These 2 generative models differ in the actual learning strategy which results typically in ...


1

Short Answer Generative networks in generative network arrangements do not learn about input images directly. Their input during training is feedback from the discriminative network. The Theory in Summary The seminal paper, Generative Adversarial Networks, Goodfellow, Pouget-Abadie, Mirza, Xu, Warde-Farley, Ozair, Courville, and Bengio, June 2014, states, ...


1

If you're building a straight "vanilla" generative adversarial network, it's best to understand the network as a statistical engine: You are training the generator on samples of a statistical distribution. (And you're training the discriminator to distinguish between "ground truth" images, and images from that generator.) Once you replace the input noise ...


1

It looks like you're asking about the difference between using conditional and joint probabilities. The joint probability $$D(x,y)$$ is the probability of x and y both happening together. The conditional probability $$D(x | y)$$ is the probability that x happens, given that y has already happened. So, $$D(x,y) = D(y) * D(x | y)$$. Notice that, in a C-GAN,...


Only top voted, non community-wiki answers of a minimum length are eligible