Monthly
288 pp. per issue
6 x 9, illustrated
ISSN
0899-7667
E-ISSN
1530-888X
2014 Impact factor:
2.21

Neural Computation

March 1994, Vol. 6, No. 2, Pages 255-269
(doi: 10.1162/neco.1994.6.2.255)
© 1994 Massachusetts Institute of Technology
The Role of Weight Normalization in Competitive Learning
Article PDF (594.11 KB)
Abstract

The effect of different kinds of weight normalization on the outcome of a simple competitive learning rule is analyzed. It is shown that there are important differences in the representation formed depending on whether the constraint is enforced by dividing each weight by the same amount (“divisive enforcement”) or subtracting a fixed amount from each weight (“subtractive enforcement”). For the divisive cases weight vectors spread out over the space so as to evenly represent “typical” inputs, whereas for the subtractive cases the weight vectors tend to the axes of the space, so as to represent “extreme” inputs. The consequences of these differences are examined.