- EM algorithm is a numerical method. It is not specific to any machine learning model. Common applications include hidden markov model and mixed Gaussians. The algorithm is not a classifier.
- Logistic regression is a statistical model. You need to pick a numerical method for logistic regression.
- Naive Bayesian is a statistical model. You need to pick a numerical method (if closed-form posterior distribution not available).
You will need to understand
maximum likelihood before you tackle the EM algorithm. Briefly, the maximum likelihood is a method for estimating the most likely parameters in your model. For instance, if you have a sequence of randomly and identically distributed Gaussian random variables, the maximum likelihood estimator for your Gaussian mean is just the sample mean.
When you fit a logistic regression, you use a numerical method (e.g. iteratively reweighted least squares) to maximise your log-likelihood function.
Everything is good, but it's not possible to maximum the likelihood directly if you have some latent variables. A common example is modelling your DNA sequences with hidden markov model, where the latent state is unknown.
You can't do it because you don't know the latent variables. If you do, they are not latent by definition.
EM algorithm is a numerical method to estimate maximum likelihood when you have latent variables. The mathematics is complicated but the idea is simple. You start off with some initial values for your parameters. You update your parameters and latent variables, and the algorithm converges when the change in the log-likelihood function falls below some threshold.