| |
Abstract:
Exact inference in densely connected Bayesian networks is
computationally intractable, and so there is considerable interest
in developing effective approximation schemes. One approach which
has been adopted is to bound the log likelihood using a mean-field
approximating distribution. While this leads to a tractable
algorithm, the mean field distribution is assumed to be factorial
and hence unimodal. In this paper we demonstrate the feasibility of
using a richer class of approximating distributions based on
mixtures of mean field distributions. We derive an efficient
algorithm for updating the mixture parameters and apply it to the
problem of learning in sigmoid belief networks. Our results
demonstrate a systematic improvement over simple mean field theory
as the number of mixture components is increased.
|
