# Tag Info

In deep networks there is actually a wide variety of solutions to the problem, but if you need to find one, any easy way to do this is just through normal optimization schemes $$\hat x = argmin_x \ L(y,x)$$ where $L(y,x)$ is your loss function. Since ANN's are generally differentiable you can optimize this iteratively with some form gradient descent scheme: $... 2 Generally, text generators work by modeling the joint distribution of the text by its Bayesian forward decomposition$ \begin{align*} p(w_1, w_2, ..., w_n) &= p(w_1) * p(w_2|w_1) * p(w_3|w_2, w_1) *\ ...\ * p(w_n|\{w_i\}_{i<n})\\ &= \prod_{i=1}^n p(w_i|\{w_k\}_{k<i})\\ \end{align*} $From a modeling perspective, this looks right up ... 2 I just came across this piece of news yesterday: "This week, Microsoft Research threw down the gauntlet with the launch of a competition challenging researchers around the world to develop AI agents that can solve text-based games." This seems to be an AI competition announced by Microsoft with the aim to create AI that can solve text-based games. This ... 2 To understand this equation first you need to understand the context in which it is first introduced. We have two neural networks (i.e.$D$and$G$) that are playing a minimax game. This means that they have competing goals. Let's look at each one separately: Generator Before we start, you should note that throughout the whole paper the notion of the data-... 2 I'll answer your questions one by one: In this equation are the$E_{z \sim p_z(z)}$and$E_{x \sim p_{data}(x)}$the means of the distributions of the mini batch samples? So let's take the first part$E_{x \sim p_{data}(x)}[log \,D(x)]$. This is read as the "expected value of$log \, D(x)$, where$x$is sampled from$p_{data}(x)$". So, in simpler terms ... 1 In general, you should train both discriminator D and generator G simultaneously. Depending on the metric that you use as the target for your model, you may encounter a Vanishing gradient problem. It can happen when you implement original loss (i.e. JS-divergence). In that case D can become overconfident regarding fake samples and won't provide any useful ... 1 I'd challenge your assertion somewhat that the generated images of other categories are of much worse quality than the faces! Take the bikes on transparent / solid backgrounds they look great! Where the images fail a bit is with the more complex pictures which have a lot of elements where element bleed (covers bleeding into the floor, etc.) occurs. This is ... 1 Generative Adversarial Networks, basically boil down to a combination of a generic Generator and a Discriminator trying to beat each other, so that the generator tries to generate much better images (usually from noise) and discriminator becomes much better at classification. So, no it is not just suited for only synthesis high quality human face synthesis ... 1 GANs are notably hard to train and it is not uncommon to have large bumps in the losses. The learning rate is a good start but the instability may come from a wide variety of reasons. I'm assuming that you have no bug in your code or data. For one, gradient descent is not well suited to the 2-player game we're playing. I've personally found ExtraAdam to ... 1 [Answering my own question after 5 months of studying VAE models] The point of the MMD-VAE or InfoVAE is not exactly to emphasise on the visual quality of generated samples. It is to preserve greater amount of information through the encoding process. The MMD formulation stems from introducing a mutual coefficient factor into the Evidence Lower BOund (ELBO) ... 1 I believe you may want to use a Sum Product Network for this task. SPNs are the state-of-the-art approach for face completion, and there are several more recent papers on this topic since the original above. Importantly, the SPN paper also covers other approaches that work well for this task. If lower-resolution results are acceptable for your task, PCA ... 1 I don't think he said that at all. Going back to the talk you'll see he mentions mode collapse comes from the naivete of using alternating gradient-based optimization steps because then$min_{\phi}max_{\theta}L(G_\phi, D_\theta)$starts to look a lot like$max_{\theta}min_{\phi}L(G_\phi, D_\theta)$. This is problematic because in the latter case the ... 1 Your mistake is that you think that the referenced$V(D,G)$is the deifinition of the cross entropy! Indeed, the cross entropy is defined base on the negative value of the$V(D,G)$. Hence, if you consider the minus behind the$V(D,G)$($-V(D,G)$) the sentence will be meaningful. 1 Your goal is to model a distribution when constructing a GAN, therefore you need a way to be able to sample that distribution. The noise's purpose is so you can do this. Generally, it's drawn from a distribution that is computationally easy to draw from (like a gaussian). You are modeling the generator$G(X)$where$X \sim N(\mu, \sigma^2)$. this means$...