1
$\begingroup$

I am currently studying Deep Learning by Goodfellow, Bengio, and Courville. In chapter 5.1.2 The Performance Measure, $P$, the authors say the following:

Unsupervised learning and supervised learning are not formally defined terms. The lines between them are often blurred. Many machine learning technologies can be used to perform both tasks. For example, the chain rule of probability states that for a vector $\mathbf{x} \in \mathbb{R}^n$, the joint distribution can be decomposed as

$$p(\mathbf{x}) = \prod_{i = 1}^n p(x_i \vert x_1, \dots, x_{i - 1} ).$$

This decomposition means that we can solve the ostensibly unsupervised problem of modeling $p(\mathbf{x})$ by splitting it into $n$ supervised learning problems. Alternatively, we can solve the supervised learning problem of learning $p(y \vert \mathbf{x})$ by using traditional unsupervised technologies to learn the joint distribution $p(\mathbf{x}, y)$, then inferring

$$p(y \vert \mathbf{x} ) = \dfrac{p(\mathbf{x}, y)}{\sum_{y'}p(\mathbf{x}, y')}.$$

I found this part vague:

Alternatively, we can solve the supervised learning problem of learning $p(y \vert \mathbf{x})$ by using traditional unsupervised technologies to learn the joint distribution $p(\mathbf{x}, y)$, then inferring

$$p(y \vert \mathbf{x} ) = \dfrac{p(\mathbf{x}, y)}{\sum_{y'}p(\mathbf{x}, y')}.$$

Can someone please elaborate on this, and also explain more clearly the role of $p(y \vert \mathbf{x} ) = \dfrac{p(\mathbf{x}, y)}{\sum_{y'}p(\mathbf{x}, y')}$?

I would greatly appreciate it if people would please take the time to clarify this.

$\endgroup$
2
$\begingroup$

This is the definition of conditional probability + Total probability decomposition formula:

$p(y|x) = \frac{p(y,x}{p(x)} = \frac{p(x,y)}{\sum_{y'}p(x,y')}$.

The idea is to use some unsupervised learning algorithm to learn the distribution $p(x,y)$ for every possible value of $y$, and by using the previous formula you can find $p(y|x)$.

| improve this answer | |
$\endgroup$
  • $\begingroup$ Thanks for the answer. What is the "previous" formula you're referring to? $\endgroup$ – The Pointer Feb 25 at 11:05
  • $\begingroup$ I pionted to p(y|x)=p(y,xp(x)=p(x,y)∑y′p(x,y′) $\endgroup$ – hola Feb 25 at 11:15
  • $\begingroup$ Ok, thanks. What is $y'$ supposed to be, as opposed to $y$? $\endgroup$ – The Pointer Feb 25 at 11:20
  • $\begingroup$ when we write a sum over y', it means that y' is ranging over all possible values of y. Consider for example the case where you have a binary classification, then y' can take values 0 and 1. In that case p(y=0|x) = p(x,y=0)/ [p(x,y=0) + p(x,y=1)] $\endgroup$ – hola Feb 25 at 11:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.