Let be a Bayesian multivariate normal distribution classifier with distinct covariance matrices for each class and isotropic, i.e. with equal values over the entire diagonal and zero otherwise, $\mathbf{\Sigma}_i=\sigma_i^2\mathbf{I},~\forall i$.

How can I compute the equation for estimating the parameter $\sigma_{i}$ by the maximum likelihood method? Here $\sigma_{i,j}$ is is the covariance between $x_i$ and $x_j$. So $\sigma_i$ is just the variance of $x_i$.


Suppose $\mathcal{X}_i = \{x^t_i\}^N_{t=1}$ i.i.d, $x_i^t$ is in the class $C_i$ and $x_i^t \sim \mathcal{N}(\mu, \sigma^2)$.

Do I have to find the log-likelihood under $p(x) = \frac{1}{\sqrt{2 \pi} \sigma} \exp \left[ -\frac{(x-\mu)^2}{2 \sigma^2}\right]$, find the derivative and put it equal to $0$ to find the maximum?


Suppose my data points are $m$-dimensional, and I have $K$ classes.

  • $\begingroup$ Comments are not for extended discussion; this conversation has been moved to chat. $\endgroup$
    – nbro
    Sep 25 '20 at 20:40
  • $\begingroup$ The problem here is how will you estimate the MLE. It is pretty difficult estimation problem (computationally) and thus has tradeoffs in space vs computational complexity. proceedings.mlr.press/v75/hazan18a/hazan18a.pdf A good place to start your problwm. Although I would warn, its a pretty difficult problem requiring a very good theoretical knowledge. $\endgroup$
    – user9947
    Sep 25 '20 at 20:55
  • $\begingroup$ How can your data points be $m$ dimensional if they follow a univariate normal distribution? $\endgroup$ Sep 25 '20 at 21:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.