Monthly
288 pp. per issue
6 x 9, illustrated
ISSN
0899-7667
E-ISSN
1530-888X
2014 Impact factor:
2.21

Neural Computation

March 2008, Vol. 20, No. 3, Pages 636-643
(doi: 10.1162/neco.2007.07-06-296)
© 2008 Massachusetts Institute of Technology
A Note on Lewicki-Sejnowski Gradient for Learning Overcomplete Representations
Article PDF (76.14 KB)
Abstract

Overcomplete representations have greater robustness in noise environment and also have greater flexibility in matching structure in the data. Lewicki and Sejnowski (2000) proposed an efficient extended natural gradient for learning the overcomplete basis and developed an overcomplete representation approach. However, they derived their gradient by many approximations, and their proof is very complicated. To give a stronger theoretical basis, we provide a brief and more rigorous mathematical proof for this gradient in this note. In addition, we propose a more robust constrained Lewicki-Sejnowski gradient.