-1
$\begingroup$

I have a sampled data, say, many vectors $m_1, \cdots, m_i \in R^{d}$.

I want an objective function that rates the probability of this data being drawn from certain distribution (here, specifically standard Gaussian $N(0, \mathbf{I}_{d\times d})$). That is to say, I want a function $f(m_1, \cdots, m_i)$ that takes the vectors $m_1, \cdots, m_i \in R^{d}$ and outputs a scalar that, the more likely the vectors are sampled from the distribution $N(0, \mathbf{I}_{d\times d})$, the smaller(or bigger) the function's return value is.

Is there any predefined ones in statistics or math theory?

I don't know much about math, just some simple ones from probabilities & statistics undergrad course.

$\endgroup$
1

2 Answers 2

0
$\begingroup$

Yes, it exists, and it's called the likelihood function (but if you want to actually calculate that "scalar", you better use the log likelihood)

$\endgroup$
-1
$\begingroup$

In general there're many ways to achieve your request in statistics such as goodness-of-fit chi-squared test, likelihood function as mentioned in another answer, KL-Divergence, etc. And perhaps in your high dimensional case, Mahalanobis distance is a good fit.

The Mahalanobis distance is a measure of the distance between a point ${\displaystyle P}$ and a distribution ${\displaystyle D}$, introduced by P. C. Mahalanobis in 1936... If each of these axes is re-scaled to have unit variance, then the Mahalanobis distance corresponds to standard Euclidean distance in the transformed space. The Mahalanobis distance is thus unitless, scale-invariant, and takes into account the correlations of the data set.

For a normal distribution in any number of dimensions, the probability density of an observation ${\displaystyle {\vec {x}}}$ is uniquely determined by the Mahalanobis distance ${\displaystyle d}$... The Mahalanobis distance is proportional, for a normal distribution, to the square root of the negative log-likelihood (after adding a constant so the minimum is at zero).

Therefore by calculating the Mahalanobis distance for each data point with respect to the mean of your standard multivariate Gaussian distribution, you can obtain a measure of how well your total sampled data points fit the multivariate Gaussian. A smaller summed value indicates a better fit to the distribution and is equivalent to negative log-likelihood essentially as explained in above reference.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .