Finite Mixture Models and the Expectation-Maximization algorithm
A finite mixture model is a way of representing the presence of subgroups within an overall population. A well-known application of such a model is the recognition of handwritten digits between 0 and 9 from scanned binary images. The population is formed by all possible images of handwritten digits and consists of 10 natural subgroups.
We would like to determine (estimate) the properties of the individual subgroups. This would be very easy if we knew which observation comes from which subgroup, i.e. if we knew the corresponding "labels". However, this is rarely the case and we face the so-called missing data problem.
What we can do is the following. We can first guess what the labels should be and then iteratively update the estimated parameters of the subgroups, update the labels, update the estimated parameters of the subgroups, update the labels, ..., until convergence. This is the basic idea of the Expectation-Maximization algorithm.
We will shortly discuss the details of the algorithm and the possible traps and pitfalls you might encounter when applying the algorithm to your dataset.