Monthly
288 pp. per issue
6 x 9, illustrated
ISSN
0899-7667
E-ISSN
1530-888X
2014 Impact factor:
2.21

Neural Computation

March 1994, Vol. 6, No. 2, Pages 334-340
(doi: 10.1162/neco.1994.6.2.334)
© 1994 Massachusetts Institute of Technology
Statistical Physics, Mixtures of Distributions, and the EM Algorithm
Article PDF (311.51 KB)
Abstract

We show that there are strong relationships between approaches to optmization and learning based on statistical physics or mixtures of experts. In particular, the EM algorithm can be interpreted as converging either to a local maximum of the mixtures model or to a saddle point solution to the statistical physics system. An advantage of the statistical physics approach is that it naturally gives rise to a heuristic continuation method, deterministic annealing, for finding good solutions.