6

The Focus of This Question "How can ... we process the data from the true distribution and the data from the generative model in the same iteration? Analyzing the Foundational Publication In the referenced page, Understanding Generative Adversarial Networks (2017), doctoral candidate Daniel Sieta correctly references Generative Adversarial Networks, ...


4

Compare generated and real data All the results produced by G are always considered "wrong" by definition, even for a very good generator. You provide the discriminative neural network $D$ with a mix of results generated by the generator network $G$ and real results from an outside source, and then you train it to distinguish if the result was produced by ...


3

It is called a Latent variable because you cannot access it during train time (which means manipulate it), In a normal Feed Forward NN you cannot manipulate the values output by hidden layers. Similarly the case here. The term originally came from RBM's (they used term hidden variables). The interpretation of hidden variables in the context of RBM was that ...


3

With Generative Adversarial Networks, all the generator cares about is fooling the discriminator. There's no requirement to be clever, or exhaustive, or make efficient use of the input space. As long as the discriminator returns "real" (vs. "fake") the generator "wins". The hope is that as the generator and discriminator are trained simultaneously, each ...


3

Let's start at the beginning. GANs are models that can learn to create data that is similar to the data that we give them. When training a generative model other than a GAN, the easiest loss function to come up with is probably the Mean Squared Error (MSE). Kindly allow me to give you an example (Trickot L 2017): Now suppose you want to generate cats ; ...


3

Not necessarily it depends on the function of the problem space for both the GANs. A real world example: a batter's reaction time and a pitchers max speed are actual bounded values based on genetics and physics. If the max speed a pitcher can pitch is greater than the max reaction time a human needs to effectively hit against them they will permanently be ...


2

A discriminative network ($D$) learns to discriminate by definition - we provide it with the true and the generated data, and let it learn by itself how to discriminate between the two. Therefore, we expect network $D$ to improve the ability of network $G$ to generate better and better images (or other kind of data), as it try to "trick" network $D$ by ...


2

in my experience GANs work really well for the scenario of semisupervised learning, where you don't necessarily have labels for all your class B data but you do have a balanced dataset. In my (limited) experience, you do have to have a balanced (in numbers) set of A and B objects, even though you are not sure of the labels. And yes, GANs can overfit to ...


2

I think you'll enjoy this work from Apple on improving the realism of synthetic images. Essentially what you need to do is generate a synthetic image then have your GAN modify the synthetic image so that a 1) a discriminator thinks it is real while also 2) not changing the gross structure of the image very much (so the traffic sign doesn't move) - yes, this ...


2

"Adversarial examples are inputs to machine learning models that an attacker has intentionally designed to cause the model to make a mistake; they’re like optical illusions for machines.". Source: "Attacking Machine Learning with Adversarial Examples" You create an input and test against the output, tuning the input to maximize the error. There are ...


2

The terms Supervised Learning and Unsupervised Learning predate the invention of the application of artificial networks to a generative and discriminative network pair, which was the first popular generative topology. The existence of labeling is the key distinction between the two. Even partial labeling indicates supervision, as odd as that jargon is, ...


2

The key is: VAE usually use a small latent dimension, the information of input is so hard to pass through this bottleneck, meanwhile it tries to minimize the loss with the batch of input data, you should know the result -- VAE can only have a mean and blurry output. If you increase the bandwidth of the bottleneck, i.e. the size of latent vector, VAE can get ...


2

In essence, Variational Autoencoders learn an "explicit" distribution of the data by trying to fit the data via a multi-dimensional Gaussian/Normal distribution. However, Generative Adversarial Networks learn an "implicit" distribution of data meaning that you cannot directly sample them. Also, due to the deterministic nature of neural networks GANs tend to ...


2

The main reason that the discriminator is trained concurrently with the generator is to provide (at least in theory) a smooth and gradual learning signal for the generator. If we trained the discriminator on only the input data, then, assuming our training algorithm converges well, it should quickly converge to a fixed model. The generator can then learn to ...


2

Latent is a synonym for hidden. Why is it called a hidden (or latent) variable? For example, suppose that you observe the behaviour of a person or animal. You can only observe the behaviour. You cannot observe the internal state (e.g. the mood) of this person or animal. The mood is a hidden variable because it cannot be observed directly (but only ...


2

To understand this equation first you need to understand the context in which it is first introduced. We have two neural networks (i.e. $D$ and $G$) that are playing a minimax game. This means that they have competing goals. Let's look at each one separately: Generator Before we start, you should note that throughout the whole paper the notion of the data-...


2

I'll answer your questions one by one: In this equation are the $E_{z \sim p_z(z)}$ and $E_{x \sim p_{data}(x)}$ the means of the distributions of the mini batch samples? So let's take the first part $E_{x \sim p_{data}(x)}[log \,D(x)]$. This is read as the "expected value of $log \, D(x)$, where $x$ is sampled from $p_{data}(x)$". So, in simpler terms ...


2

To put it simply GANs suffer from a problem of uneven learning rate. Imagine the learning rate of a pitcher and hitter if the pitcher gets to a point where they can throw much better than the hitter can hit then the hitter may fall into a 'training pit' as to be unable to ever learn how to hit from the pitcher. This follows a continues relationship in ...


2

Why AI is (or not) a good option for the generation of random numbers? AI approaches are generally not good for generating random numbers, for these reasons: Similar to why they are not good for adding numbers, there already exist many strong pseudo-random and "true" random sources, possible without using any AI approach, and demonstrably good enough for ...


2

The only disadvantage and difference between these generative models and the method you describe, is the input. You describe inputting tags, where as for a GAN, or VAE, the generation segment of the model takes in some representation of a probability distribution. For a GAN, it's mostly random noise, and for a VAE it is some latent space (see nbros answer). ...


2

What's the input to the Generator? In the basic implementation of GANs, the Generator only takes in a vector of random variables. This might seem strange, but after training, the generator can transform this input noise into an image resembling those of the training set. How does it work? It is trained along with its counterpart the Discriminator, whose ...


1

I don't think he said that at all. Going back to the talk you'll see he mentions mode collapse comes from the naivete of using alternating gradient-based optimization steps because then $min_{\phi}max_{\theta}L(G_\phi, D_\theta)$ starts to look a lot like $max_{\theta}min_{\phi}L(G_\phi, D_\theta)$. This is problematic because in the latter case the ...


1

Lets start with question 1) how does JS-divergence handles zeros? by definition: \begin{align} D_{JS}(p||q) &= \frac{1}{2}[D_{KL}(p||\frac{p+q}{2}) + D_{KL}(q||\frac{p+q}{2})] \\ &= \frac{1}{2}\sum_{x\in\Omega} [p(x)log(\frac{2 p(x)}{p(x)+q(x)}) + q(x)log(\frac{2 q(x)}{p(x)+q(x)})] \end{align} Where $\Omega$ is the union of the domains of $p$ and ...


1

Your mistake is that you think that the referenced $V(D,G)$ is the deifinition of the cross entropy! Indeed, the cross entropy is defined base on the negative value of the $V(D,G)$. Hence, if you consider the minus behind the $V(D,G)$ ($-V(D,G)$) the sentence will be meaningful.


1

The model (that I know of) which most resembles your description is the auto-encoder, which is trained to learn a compact representation (a vector) of the input, which can later be used to reconstruct the original input. In a certain way, this compact representation (implicitly) encodes the most important features of the input. In particular, you may be ...


1

Your goal is to model a distribution when constructing a GAN, therefore you need a way to be able to sample that distribution. The noise's purpose is so you can do this. Generally, it's drawn from a distribution that is computationally easy to draw from (like a gaussian). You are modeling the generator $G(X)$ where $X \sim N(\mu, \sigma^2)$. this means $...


1

It's not supposed to be derived from some equation. That is the basic premise under which GANs work. The output of the Generator $G(z)$ is fed as an input $x_g$ to the discriminator.


1

As @Clement mentions, text_gen_description gives a good overview!, but the paper seqGAN paper describes the REINFORCE approach more in depth, as they are the first to do it (i believe). This is probably the approach most take now of days when going the GAN route. Note that just basic MLE training has shown promise with openAI's GPT2. When i need a text ...


1

With a Google Cloud V100 GPU the GAN would run a week to two with default parameters. Does this sound realistic time for this kind of dataset? It's definitely not feasible for me. Yes, V100s are quite beefy. You shouldn't even need a week. Obviously this is based on my experience with various problems, rather than a concrete calculation. Is 4000 ...


1

Let me first provide a brief introduction first GANs as VAEs are generative models which means they learn exactly what you described: to map a typically small dimensional vector/tensor into a higher dimensional one, which in your case is (interpreted as) an image. These 2 generative models differ in the actual learning strategy which results typically in ...


Only top voted, non community-wiki answers of a minimum length are eligible