7

The Focus of This Question "How can ... we process the data from the true distribution and the data from the generative model in the same iteration? Analyzing the Foundational Publication In the referenced page, Understanding Generative Adversarial Networks (2017), doctoral candidate Daniel Sieta correctly references Generative Adversarial Networks, ...


4

Compare generated and real data All the results produced by G are always considered "wrong" by definition, even for a very good generator. You provide the discriminative neural network $D$ with a mix of results generated by the generator network $G$ and real results from an outside source, and then you train it to distinguish if the result was produced by ...


3

Latent is a synonym for hidden. Why is it called a hidden (or latent) variable? For example, suppose that you observe the behaviour of a person or animal. You can only observe the behaviour. You cannot observe the internal state (e.g. the mood) of this person or animal. The mood is a hidden variable because it cannot be observed directly (but only ...


3

It is called a Latent variable because you cannot access it during train time (which means manipulate it), In a normal Feed Forward NN you cannot manipulate the values output by hidden layers. Similarly the case here. The term originally came from RBM's (they used term hidden variables). The interpretation of hidden variables in the context of RBM was that ...


3

I think you'll enjoy this work from Apple on improving the realism of synthetic images. Essentially what you need to do is generate a synthetic image then have your GAN modify the synthetic image so that a 1) a discriminator thinks it is real while also 2) not changing the gross structure of the image very much (so the traffic sign doesn't move) - yes, this ...


3

The key is: VAE usually use a small latent dimension, the information of input is so hard to pass through this bottleneck, meanwhile it tries to minimize the loss with the batch of input data, you should know the result -- VAE can only have a mean and blurry output. If you increase the bandwidth of the bottleneck, i.e. the size of latent vector, VAE can get ...


3

In essence, Variational Autoencoders learn an "explicit" distribution of the data by trying to fit the data via a multi-dimensional Gaussian/Normal distribution. However, Generative Adversarial Networks learn an "implicit" distribution of data meaning that you cannot directly sample them. Also, due to the deterministic nature of neural networks GANs tend to ...


3

With Generative Adversarial Networks, all the generator cares about is fooling the discriminator. There's no requirement to be clever, or exhaustive, or make efficient use of the input space. As long as the discriminator returns "real" (vs. "fake") the generator "wins". The hope is that as the generator and discriminator are trained simultaneously, each ...


3

Let's start at the beginning. GANs are models that can learn to create data that is similar to the data that we give them. When training a generative model other than a GAN, the easiest loss function to come up with is probably the Mean Squared Error (MSE). Kindly allow me to give you an example (Trickot L 2017): Now suppose you want to generate cats ; ...


3

Not necessarily it depends on the function of the problem space for both the GANs. A real world example: a batter's reaction time and a pitchers max speed are actual bounded values based on genetics and physics. If the max speed a pitcher can pitch is greater than the max reaction time a human needs to effectively hit against them they will permanently be ...


3

In reality GANs are not made for image classification, but for data generation, and they have gained popularity on image generation. They are also used for tabular data generation, see for example TGAN, or for time series generation, e.g. Quant GAN. You have even some application for the field of graphs and networking, e.g. NetGAN and GraphGAN.


2

A discriminative network ($D$) learns to discriminate by definition - we provide it with the true and the generated data, and let it learn by itself how to discriminate between the two. Therefore, we expect network $D$ to improve the ability of network $G$ to generate better and better images (or other kind of data), as it try to "trick" network $D$ by ...


2

in my experience GANs work really well for the scenario of semisupervised learning, where you don't necessarily have labels for all your class B data but you do have a balanced dataset. In my (limited) experience, you do have to have a balanced (in numbers) set of A and B objects, even though you are not sure of the labels. And yes, GANs can overfit to ...


2

It looks like you're asking about the difference between using conditional and joint probabilities. The joint probability $$D(x,y)$$ is the probability of x and y both happening together. The conditional probability $$D(x | y)$$ is the probability that x happens, given that y has already happened. So, $$D(x,y) = D(y) * D(x | y)$$. Notice that, in a C-GAN,...


2

"Adversarial examples are inputs to machine learning models that an attacker has intentionally designed to cause the model to make a mistake; they’re like optical illusions for machines.". Source: "Attacking Machine Learning with Adversarial Examples" You create an input and test against the output, tuning the input to maximize the error. There are ...


2

The terms Supervised Learning and Unsupervised Learning predate the invention of the application of artificial networks to a generative and discriminative network pair, which was the first popular generative topology. The existence of labeling is the key distinction between the two. Even partial labeling indicates supervision, as odd as that jargon is, ...


2

To understand this equation first you need to understand the context in which it is first introduced. We have two neural networks (i.e. $D$ and $G$) that are playing a minimax game. This means that they have competing goals. Let's look at each one separately: Generator Before we start, you should note that throughout the whole paper the notion of the data-...


2

The main reason that the discriminator is trained concurrently with the generator is to provide (at least in theory) a smooth and gradual learning signal for the generator. If we trained the discriminator on only the input data, then, assuming our training algorithm converges well, it should quickly converge to a fixed model. The generator can then learn to ...


2

I'll answer your questions one by one: In this equation are the $E_{z \sim p_z(z)}$ and $E_{x \sim p_{data}(x)}$ the means of the distributions of the mini batch samples? So let's take the first part $E_{x \sim p_{data}(x)}[log \,D(x)]$. This is read as the "expected value of $log \, D(x)$, where $x$ is sampled from $p_{data}(x)$". So, in simpler terms ...


2

To put it simply GANs suffer from a problem of uneven learning rate. Imagine the learning rate of a pitcher and hitter if the pitcher gets to a point where they can throw much better than the hitter can hit then the hitter may fall into a 'training pit' as to be unable to ever learn how to hit from the pitcher. This follows a continues relationship in ...


2

Why AI is (or not) a good option for the generation of random numbers? AI approaches are generally not good for generating random numbers, for these reasons: Similar to why they are not good for adding numbers, there already exist many strong pseudo-random and "true" random sources, possible without using any AI approach, and demonstrably good enough for ...


2

The only disadvantage and difference between these generative models and the method you describe, is the input. You describe inputting tags, where as for a GAN, or VAE, the generation segment of the model takes in some representation of a probability distribution. For a GAN, it's mostly random noise, and for a VAE it is some latent space (see nbros answer). ...


2

What's the input to the Generator? In the basic implementation of GANs, the Generator only takes in a vector of random variables. This might seem strange, but after training, the generator can transform this input noise into an image resembling those of the training set. How does it work? It is trained along with its counterpart the Discriminator, whose ...


2

It might be that your dataset of images is to small. Your discriminative network might hardlearn these images at which point your generative network can only produce good images if it copies the same images of your dataset.


2

In full: The limit, as standard deviation $\sigma$ tends towards zero, of the gradient with respect to vector $\mathbf{x}$, of the expectation - where perturbation $\epsilon$ follows the normal distribution with mean 0 and variance $\sigma^2$ times identity vector $[1,1,1,1...]$ * - of any function $f$ of $\mathbf{x}$ plus $\epsilon$ is equal to the ...


1

I don't think he said that at all. Going back to the talk you'll see he mentions mode collapse comes from the naivete of using alternating gradient-based optimization steps because then $min_{\phi}max_{\theta}L(G_\phi, D_\theta)$ starts to look a lot like $max_{\theta}min_{\phi}L(G_\phi, D_\theta)$. This is problematic because in the latter case the ...


1

Lets start with question 1) how does JS-divergence handles zeros? by definition: \begin{align} D_{JS}(p||q) &= \frac{1}{2}[D_{KL}(p||\frac{p+q}{2}) + D_{KL}(q||\frac{p+q}{2})] \\ &= \frac{1}{2}\sum_{x\in\Omega} [p(x)log(\frac{2 p(x)}{p(x)+q(x)}) + q(x)log(\frac{2 q(x)}{p(x)+q(x)})] \end{align} Where $\Omega$ is the union of the domains of $p$ and ...


1

Your mistake is that you think that the referenced $V(D,G)$ is the deifinition of the cross entropy! Indeed, the cross entropy is defined base on the negative value of the $V(D,G)$. Hence, if you consider the minus behind the $V(D,G)$ ($-V(D,G)$) the sentence will be meaningful.


1

The model (that I know of) which most resembles your description is the auto-encoder, which is trained to learn a compact representation (a vector) of the input, which can later be used to reconstruct the original input. In a certain way, this compact representation (implicitly) encodes the most important features of the input. In particular, you may be ...


1

Your goal is to model a distribution when constructing a GAN, therefore you need a way to be able to sample that distribution. The noise's purpose is so you can do this. Generally, it's drawn from a distribution that is computationally easy to draw from (like a gaussian). You are modeling the generator $G(X)$ where $X \sim N(\mu, \sigma^2)$. this means $...


Only top voted, non community-wiki answers of a minimum length are eligible