0
$\begingroup$

In probability, we have two types of probability functions: unconditional probability $p(x)$ and conditional probability $p(x | y)$. Both are fundamentally different and the latter can be obtained by the following equation

$$p(x|y) = \dfrac{p(x, y)}{p(y)} \text{ provided } p(y) \ne 0$$

I never heard of formal definition for conditioning except for conditional probability function.

But in case of neural networks, I came across the notion of conditioning.

$$\min_G \max_DV(D, G) = \mathbb{E}_{x ∼ P_{data}}[\log D(x|y)] + \mathbb{E}_{z ∼ p_z}[log (1 - D(G(z|y)))]$$

Since neural network $D$ is intended to implement a probability function, we can at-least think about conditioning on an input. But the neural network $G$ is not intended to implement probability function. $G$ is intended to provide datasamples by learning an underlying probability distribution whose output is not in the range $[0, 1]$.

Does $G$ obey the laws of probability? If yes, how, since its output is not restricted to $[0, 1]$? If no, then why the authors use the notation of conditional probability for $G$ also?

$\endgroup$
4
  • 1
    $\begingroup$ Why would the output have to be in the range [0,1]? Who said G's output is a probability? $\endgroup$
    – user253751
    Aug 5 at 10:36
  • 1
    $\begingroup$ G is more like a function for sampling from a probability distribution $\endgroup$
    – user253751
    Aug 5 at 10:36
  • $\begingroup$ @user253751 Confusion is due to the notation of conditional probability $G(x|y)$. $\endgroup$
    – hanugm
    Aug 5 at 10:46
  • $\begingroup$ @user253751 I didn't see any function till now of the form f(x|y) except conditional probability distribution function p(x|y). $\endgroup$
    – hanugm
    Aug 5 at 10:53
1
$\begingroup$

I don't see where it's implied that G is a probability distribution. G is a function, whose output conditioned on one variable has a probability distribution, but it isn't one.

z is random noise which is G's source of randomness. y is something that isn't random. We call G over and over with the same y and different random z's and look at the distribution of the output.

For example, here are some outputs of one possible G(z,y), for different z, when y=3. z is not displayed, only G(z,y). This G outputs one digit.

2262662626626262626262626626262262622226626622626662626666262626262626262

These digits have a probability distribution. About half of them are 2 and about half of them are 6. P(G(z)=2|y=3) = 0.479 and P(G(z)=6|y=3) = 0.521 (approximately). Even though neither 2 nor 6 is a valid probability.

$\endgroup$
2
  • $\begingroup$ So, the notation has to be $G(z,y)$, Then what might be the reason for using $G(z|y)$? $\endgroup$
    – hanugm
    Aug 5 at 12:13
  • 1
    $\begingroup$ @hanugm Notation does not matter. Notation does not have to be a certain way. It seems that both z and y are inputs to G. They probably write G(z|y) because it reminds you of conditional probability. $\endgroup$
    – user253751
    Aug 5 at 13:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.