| |
Abstract:
Prioritized sweeping is a model-based reinforcement learning
method that attempts to focus an agent's limited computational
resources to achieve a good estimate of the value of environment
states. The classic account of prioritized sweeping uses an
explicit, state-based representation of the value, reward, and
model parameters. Such a representation is unwieldy for dealing
with complex environments and there is growing interest in learning
with more compact representations. We claim that classic
prioritized sweeping is ill-suited for learning with such
representations. To overcome this deficiency, we introduce
generalized prioritized sweeping, a principled method for
generating representation-specific algorithms for model-based
reinforcement learning. We then apply this method for several
representations, including state-based models and generalized model
approximators (such as Bayesian networks). We describe preliminary
experiments that compare our approach with classical prioritized
sweeping.
|
