| |
Abstract:
Planning and learning at multiple levels of temporal
abstraction is a key problem for for artificial intelligence. In
this paper we summarize an approach to this problem based on the
mathematical framework of Markov decision processes and
reinforcement learning. Current model-based reinforcement learning
is based on one-step models that cannot represent common-sense
higher-level actions, such as going to lunch, grasping an object,
or flying to Denver. This paper generalizes prior work on
temporally abstract models (Sutton, 1995) and extends it from the
prediction setting to include actions, control, and planning. We
introduce a more general form of temporally abstract model, the
multi-time model, and establish its suitability for planning and
learning by virtue of its relationship to Bellman equations. This
paper summarizes the theoretical framework of multi-time models and
illustrates their potential advantages in a gridworld planning
task.
|
