0
$\begingroup$

In probability, we have two types of probability functions: unconditional probability $p(x)$ and conditional probability $p(x | y)$. Both are fundamentally different and the latter can be obtained by the following equation

$$p(x|y) = \dfrac{p(x, y)}{p(y)} \text{ provided } p(y) \ne 0$$

I never heard of formal definition for conditioning except for conditional probability function.

But in case of neural networks, I came across the notion of conditioning.

$$\min_G \max_DV(D, G) = \mathbb{E}_{x ∼ P_{data}}[\log D(x|y)] + \mathbb{E}_{z ∼ p_z}[log (1 - D(G(z|y)))]$$

Since neural network $D$ is intended to implement a probability function, we can at-least think about conditioning on an input. But the neural network $G$ is not intended to implement probability function. $G$ is intended to provide datasamples by learning an underlying probability distribution whose output is not in the range $[0, 1]$.

Does $G$ obey the laws of probability? If yes, how, since its output is not restricted to $[0, 1]$? If no, then why the authors use the notation of conditional probability for $G$ also?

$\endgroup$
4
  • 1
    $\begingroup$ Why would the output have to be in the range [0,1]? Who said G's output is a probability? $\endgroup$
    – user253751
    Aug 5, 2021 at 10:36
  • 1
    $\begingroup$ G is more like a function for sampling from a probability distribution $\endgroup$
    – user253751
    Aug 5, 2021 at 10:36
  • $\begingroup$ @user253751 Confusion is due to the notation of conditional probability $G(x|y)$. $\endgroup$
    – hanugm
    Aug 5, 2021 at 10:46
  • $\begingroup$ @user253751 I didn't see any function till now of the form f(x|y) except conditional probability distribution function p(x|y). $\endgroup$
    – hanugm
    Aug 5, 2021 at 10:53

1 Answer 1

1
$\begingroup$

I don't see where it's implied that G is a probability distribution. G is a function, whose output conditioned on one variable has a probability distribution, but it isn't one.

z is random noise which is G's source of randomness. y is something that isn't random. We call G over and over with the same y and different random z's and look at the distribution of the output.

For example, here are some outputs of one possible G(z,y), for different z, when y=3. z is not displayed, only G(z,y). This G outputs one digit.

2262662626626262626262626626262262622226626622626662626666262626262626262

These digits have a probability distribution. About half of them are 2 and about half of them are 6. P(G(z)=2|y=3) = 0.479 and P(G(z)=6|y=3) = 0.521 (approximately). Even though neither 2 nor 6 is a valid probability.

$\endgroup$
2
  • $\begingroup$ So, the notation has to be $G(z,y)$, Then what might be the reason for using $G(z|y)$? $\endgroup$
    – hanugm
    Aug 5, 2021 at 12:13
  • 1
    $\begingroup$ @hanugm Notation does not matter. Notation does not have to be a certain way. It seems that both z and y are inputs to G. They probably write G(z|y) because it reminds you of conditional probability. $\endgroup$
    – user253751
    Aug 5, 2021 at 13:27

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .