Monthly
288 pp. per issue
6 x 9, illustrated
ISSN
0899-7667
E-ISSN
1530-888X
2014 Impact factor:
2.21

Neural Computation

Fall 1989, Vol. 1, No. 3, Pages 402-411
(doi: 10.1162/neco.1989.1.3.402)
© 1989 Massachusetts Institute of Technology
How to Generate Ordered Maps by Maximizing the Mutual Information between Input and Output Signals
Article PDF (697.44 KB)
Abstract

A learning rule that performs gradient ascent in the average mutual information between input and an output signal is derived for a system having feedforward and lateral interactions. Several processes emerge as components of this learning rule: Hebb-like modification, and cooperation and competition among processing nodes.

Topographic map formation is demonstrated using the learning rule. An analytic expression relating the average mutual information to the response properties of nodes and their geometric arrangement is derived in certain cases. This yields a relation between the local map magnification factor and the probability distribution in the input space. The results provide new links between unsupervised learning and information-theoretic optimization in a system whose properties are biologically motivated.