Towards Learning Hierarchical Skills for Multi-Phase Manipulation Tasks

Oliver Kroemer, Christian Daniel, Gerhard Neumann, Herke van Hoof and Jan Peters
Conference Paper, International Conference on Robotics and Automation (ICRA), January, 2015

Download Publication

Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.


Most manipulation tasks can be decomposed into a sequence of phases, where the robot’s actions have different effects in each phase. The robot can perform actions to transition between phases and, thus, alter the effects of its actions, e.g. grasp an object in order to then lift it. The robot can thus reach a phase that affords the desired manipulation.

In this paper, we present an approach for exploiting the phase structure of tasks in order to learn manipulation skills more efficiently. Starting with human demonstrations, the robot learns a probabilistic model of the phases and the phase transitions. The robot then employs model-based reinforcement learning to create a library of motor primitives for transitioning between phases. The learned motor primitives generalize to new situations and tasks. Given this library, the robot uses a value function approach to learn a high-level policy for sequencing the motor primitives. The proposed method was successfully evaluated on a real robot performing a bimanual grasping task.

author = {Oliver Kroemer and Christian Daniel and Gerhard Neumann and Herke van Hoof and Jan Peters},
title = {Towards Learning Hierarchical Skills for Multi-Phase Manipulation Tasks},
booktitle = {International Conference on Robotics and Automation (ICRA)},
year = {2015},
month = {January},
} 2019-03-12T13:50:00-04:00