0
$\begingroup$

When I was reading about discriminative vs generative models, I came across their definitions:

Given a distribution of inputs $X$ and labels $Y:$

Discriminative models learn the conditional distribution $P(Y|X)$.

Generative models learn the joint distribution $P(X,Y)$. If you only have $X$, then you can still learn $P(X)$.

My questions are:

  • What does it mean to "learn a distribution" ? Learning what from the distribution ?
  • What does the distribution contain, and what does it look like?
Improve this question
$\endgroup$

1 Answer 1

Reset to default
2
$\begingroup$

Learning the distribution: When we talk about learning a distribution, we are essentially trying to capture the underlying statistical properties of the data. In other words, we try to capture the distribution from which the data points in our dataset our sampled. This involves estimating parameters that define the distribution (such as mean, variance, etc.) or learning a model that can generate data points similar to those observed in the dataset.

What does the distribution contain?: The distribution contains information about the likelihood of different values or configurations of the variables in the dataset. For example, in a simple case where X represents the features of a dataset and Y represents the labels, the conditional distribution P(Y|X) would describe the probability of observing a particular label given the input features.

What does the distribution look like?: Depends on the nature of the data and the relationships between variables. It could take various forms, such as Gaussian (bell-shaped), uniform, exponential, etc., depending on the characteristics of the data.

Improve this answer
$\endgroup$
5
  • $\begingroup$ Thank you. What do you mean by "data points"? If I have a dataset of images, what does a data point represent for each of these images ? $\endgroup$
    – user77925
    Commented Mar 7 at 12:35
  • $\begingroup$ One datapoint would be tje combination of (x, y), so the image with label. $\endgroup$
    – Robin van Hoorn
    Commented Mar 7 at 12:55
  • $\begingroup$ Very newbie question, what is "the combination of $(x,y)$", and what do you mean by so the image with label ? $\endgroup$
    – user77925
    Commented Mar 7 at 14:21
  • $\begingroup$ @abcd If you are interested in a binary classification task, classifying whether your images have cats in them or not, then $x$ would be an image and $y$ would be the 'label', which in this example would be a 1 or 0, denoting whether the image contained a cat, or not, respectively. $\endgroup$
    – David
    Commented Mar 7 at 20:01
  • $\begingroup$ So you're saying that the data points are just number between $0$ and $1$ for classification task (discriminative model) ? $\endgroup$
    – user77925
    Commented Mar 8 at 18:45

You must log in to answer this question.