Relation between Gaussian mixture models and maximum likelihood?

Question

I need some help understanding the relation between the maximum likelihood and Gaussian mixture models. I have seen that there is a relationship between the expectation maximization algorithms and these two terms, but I could not quite understand the relation well enough. Could some please give me an example of how these two terms are related or different?

Answer 1

Maximum Likelihood is a parameter estimation method, where you select the parameters $\theta$ that maximize the Likelihood function:

$\max_\theta \sum_{i=1}^{n} f(x_i|\theta)$

Gaussian Mixture Model just means to specify the density to be fitted as some weighted average of Normal distributions $X_i\sim N(\mu_i,\sigma_i)$ (so you have more parameters from each mixed density, a "parameter vector"):

$X=\sum w_i X_i$

You may further complicate this approach by letting the weights (which sum to 1, so can be a probability) also randomized as some distribution $W_i\sim W$:

$X=\sum W_i X_i$

With such distribution, you may achieve a better fit since you can adjust the marginals and the weights altogether.

You may then apply this model of the density for the Expectation Maximization algorithm, or for Maximum Likelihood aswell. So the Gaussian Mixture Model just means to use a Gaussian Mixture distribution as the target density to be fitted to a sample by Maximum Likelihood or Expecation Maximization method.

Also see https://www.ll.mit.edu/mission/communications/ist/publications/0802_Reynolds_Biometrics-GMM.pdf