People who code: we want your input. Take the Survey

Tag Info

6

The probability density is used to 'measure how good' the parameters are because it is a natural way of quantifying if these parameters are good for the observed data. Also, as the notation often causes some confusion, $L(\theta | x)$ denotes the probability of all of your observed data, not just one value. Also the "$|$" may cause confusion as it ...

4

It seems your question is concerned with how an empirical mean works. It is indeed true that, if all $x^{(i)}$ are independent identically distributed realisations of a random variable $X$, then $\lim_{n \rightarrow \infty} \frac{1}{n}\sum_{i=1}^n f(x^{(i)}) = \mathbb{E}[f(X)]$. This is a standard result in statistics known as the law of large numbers.

2

This question is very general in the sense that the reason may differ depending on the area of ML you are considering. Below are two different areas of ML where the KL-divergence is a natural consequence: Classification: maximizing the log-likelihood (or minimizing the negative log-likelihood) is equivalent to minimizing KL divergence as typical used in DL-...

2

The idea behind this kind of reasoning is that there is a "true" distribution (unknown to us, mere mortals) and that the data is generated following this distribution. But what we don't really know the shape of the distribution, all we know is the distribution of the data that we have. This is called the empirical distribution. Let's see a simple ...

2

@The Pointer the $2^n$ came from the question: How many function do we need to have if each of the $n$ inputs can be missing? example: $f_1(\text{missing}, x_2, x_3, \dots, x_n)$ for $x_1$ missing $f_2(x_1, x_2, \text{missing}, x_4, \text{missing}, \dots, x_n)$ for $x_3$ and $x_5$ missing. So this problem is a combinatorial one and the event for each $x_i$ ...

1

Yes, due to this issue, you should use temperature scaling after training your model. It will calibrate your probability and you will start to get the same kind of distributions. Here are a good article along with implementation on it.

1

In ML we always deal with unknown probability distributions from which the data comes. The most common way to calculate the distance between real and model distribution is $KL$ divergence. Why Kullback–Leibler divergence? Although there are other loss functions (e.g. MSE, MAE), $KL$ divergence is natural when we are dealing with probability distributions. It ...

1

Consider the case of binary classification, i.e. you want to classify each input $x$ into one of two classes: $y_1$ or $y_2$. For example, in the context of object classification, $y_1$ could be "cat" and $y_2$ could be "dog", and $x$ is an image that contains one main object. In certain cases, $x$ cannot be easily classified. For example,...

1

To add to nbro's answer, I'd say also that much of the time the distance measure isn't simply a design decision, rather it comes up naturally from the model of the problem. For instance, minimizing the KL divergence between your policy and the softmax of the Q values at a given state is equivalent to policy optimization where the optimality at a given state ...

1

I did not read those two specified linked/cited papers and I am not currently familiar with the total variation distance, but I think I can answer some of your questions, given that I am reasonably familiar with the KL divergence. When you compute the $D_{KL}$ between two polices, what does that tell you about them The KL divergence is a measure of "...

1

If you're using a discount factor less than 1, you should be able to compute a maximum return (likewise, a minimum return) based on the max (min) reward you can earn at each timestep. However, this issue you bring up is usually cited as a difficulty with C51. I think people tend to simply use fixed values for the min/max return (or just make rough estimates)....

Only top voted, non community-wiki answers of a minimum length are eligible