Solving the supervised learning problem of learning $p(y \vert \mathbf{x})$ by using traditional unsupervised technologies to learn $p(\mathbf{x}, y)$

Question

I am currently studying Deep Learning by Goodfellow, Bengio, and Courville. In chapter 5.1.2 The Performance Measure, $P$, the authors say the following:

Unsupervised learning and supervised learning are not formally defined terms. The lines between them are often blurred. Many machine learning technologies can be used to perform both tasks. For example, the chain rule of probability states that for a vector $\mathbf{x} \in \mathbb{R}^n$, the joint distribution can be decomposed as

$$p(\mathbf{x}) = \prod_{i = 1}^n p(x_i \vert x_1, \dots, x_{i - 1} ).$$

This decomposition means that we can solve the ostensibly unsupervised problem of modeling $p(\mathbf{x})$ by splitting it into $n$ supervised learning problems. Alternatively, we can solve the supervised learning problem of learning $p(y \vert \mathbf{x})$ by using traditional unsupervised technologies to learn the joint distribution $p(\mathbf{x}, y)$, then inferring

$$p(y \vert \mathbf{x} ) = \dfrac{p(\mathbf{x}, y)}{\sum_{y'}p(\mathbf{x}, y')}.$$

I found this part vague:

Alternatively, we can solve the supervised learning problem of learning $p(y \vert \mathbf{x})$ by using traditional unsupervised technologies to learn the joint distribution $p(\mathbf{x}, y)$, then inferring

$$p(y \vert \mathbf{x} ) = \dfrac{p(\mathbf{x}, y)}{\sum_{y'}p(\mathbf{x}, y')}.$$

Can someone please elaborate on this, and also explain more clearly the role of $p(y \vert \mathbf{x} ) = \dfrac{p(\mathbf{x}, y)}{\sum_{y'}p(\mathbf{x}, y')}$?

I would greatly appreciate it if people would please take the time to clarify this.

Answer 1

This is the definition of conditional probability + Total probability decomposition formula:

$p(y|x) = \frac{p(y,x}{p(x)} = \frac{p(x,y)}{\sum_{y'}p(x,y')}$.

The idea is to use some unsupervised learning algorithm to learn the distribution $p(x,y)$ for every possible value of $y$, and by using the previous formula you can find $p(y|x)$.