| |
Abstract:
Online learning is one of the most common forms of neural
network training. We present an analysis of online learning from
finite training sets for non-linear networks (namely,
soft-committee machines), advancing the theory to more realistic
learning scenarios. Dynamical equations are derived for an
appropriate set of order parameters; these are exact in the
limiting case of either linear networks or infinite training sets.
Preliminary comparisons with simulations suggest that the theory
captures some effects of finite training sets, but may not yet
account correctly for the presence of local minima.
|
