Neural Computation
January 1995, Vol. 7, No. 1, Pages 13-18
(doi: 10.1162/neco.1995.7.1.13)
The EM Algorithm and Information Geometry in Neural Network Learning
Article PDF (280.66 KB)
Abstract
Hidden units play an important role in neural networks, although their activation values are unknown in many learning situations. The EM algorithm (statistical algorithm) and the em algorithm (information-geometric one) have been proposed so far in this connection, and the effectiveness of such algorithms is recognized in many areas of research. The present note points out that these two algorithms are equivalent under a certain condition, although they are different in general.