2
$\begingroup$

I'm reading the article about generative model from Open AI, here is the section where they explain them:

Our network is a function with parameters $\theta$, and tweaking these parameters will tweak the generated distribution of images. Our goal then is to find parameters $\theta$ that produce distribution that closely matches the true data distribution. Therefore, you can imagine the green distribution starting out random and then the training process iteratively changing the parameters $\theta$ to stretch and squeeze it to better match the blue distribution.

enter image description here

I don't understand what are the meaning of these parameters $\theta$ and how does tweaking these parameters will tweak the generated distribution of images, can someone explain ? Thanks!

Improve this question
$\endgroup$

1 Answer 1

Reset to default
2
$\begingroup$

As explained in your reference the yellow generative model is a neural network that takes randomly sampled points from a unit Gaussian distribution as input and generates an approximated distribution $\hat p{_\theta}(x)$ to the true data distribution $p(x)$. The unit Gaussian distribution is a simplified assumption here and it can be any latent variables $z$ aka codes of DCGANs or VAEs.

The parameters $θ$ of the generative model are the weights and biases of this generator or decoder neural network for GANs or VAEs, respectively. By adjusting the values of $θ$ you are essentially modifying how the neural network transforms input into the generated synthetic data. Learning in a generative model involves adjusting the parameters $θ$ so that the synthetic data generated by the model becomes more similar to the true data (images) distribution. This is typically done by optimizing an objective function such as ELBO, which includes terms that encourage the generated distribution to match the true data distribution.

Improve this answer
$\endgroup$
4
  • $\begingroup$ So the parameter $\theta$ are just a bunch of weights, what do you mean by weights ? $\endgroup$
    – user79662
    Commented Feb 5 at 20:51
  • $\begingroup$ They're just the commonly referred weights between nodes of fully connected layers and biases of each node in a generic MLP, they're what the backprop algo aims to settle. $\endgroup$
    – cinch
    Commented Feb 5 at 21:00
  • $\begingroup$ Very newbie question, at the start how are these weights initialized ? $\endgroup$
    – user79662
    Commented Feb 5 at 21:11
  • $\begingroup$ There're many tricks, usually they're randomly initialized to some small values to have optimal result from many experiments. $\endgroup$
    – cinch
    Commented Feb 5 at 21:13

You must log in to answer this question.