Identifying Useful Subgoals in Reinforcement Learning by Local Graph Partitioning

Özgür Şimşek, University of Massachusetts - Amherst
Alicia P. Wolfe, University of Massachusetts - Amherst
Andrew G. Barto, University of Massachusetts - Amherst

Article comments

This paper was harvested from CiteSeer

Abstract

We present a new subgoal-based method for automatically creating useful skills in reinforcement learning. Our method identifies subgoals by partitioning local state transition graphs—those that are constructed using only the most recent experiences of the agent. The local scope of our subgoal discovery method allows it to successfully identify the type of subgoals we seek—states that lie between two densely-connected regions of the state space—while producing an algorithm with low computational cost.