Monthly
288 pp. per issue
6 x 9, illustrated
ISSN
0899-7667
E-ISSN
1530-888X
2014 Impact factor:
2.21

Neural Computation

Winter 1989, Vol. 1, No. 4, Pages 552-558
(doi: 10.1162/neco.1989.1.4.552)
© 1989 Massachusetts Institute of Technology
A Subgrouping Strategy that Reduces Complexity and Speeds Up Learning in Recurrent Networks
Article PDF (370.14 KB)
Abstract

An algorithm, called RTRL, for training fully recurrent neural networks has recently been studied by Williams and Zipser (1989a, b). Whereas RTRL has been shown to have great power and generality, it has the disadvantage of requiring a great deal of computation time. A technique is described here for reducing the amount of computation required by RTRL without changing the connectivity of the networks. This is accomplished by dividing the original network into subnets for the purpose of error propagation while leaving them undivided for activity propagation. An example is given of a 12-unit network that learns to be the finite-state part of a Turing machine and runs 10 times faster using the subgrouping strategy than the original algorithm.