Learning Robust Manipulation Strategies with Multimodal State Transition Models and Recovery Heuristics

Austin S. Wang and Oliver Kroemer
Conference Paper, International Conference on Robotics and Automation (ICRA), January, 2019

Download Publication

Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.


Robots are prone to making mistakes when performing manipulation tasks in unstructured environments. Robust policies are thus needed to not only avoid mistakes but also to recover from them. We propose a framework for increasing the robustness of contact-based manipulations by modeling the task structure and optimizing a policy for selecting skills and recovery skills. A multimodal state transition model is acquired based on the contact dynamics of the task and the observed transitions. A policy is then learned from the model using reinforcement learning. The policy is incrementally improved by expanding the action space by generating recovery skills with a heuristic. Evaluations on three simulated manipulation tasks demonstrate the effectiveness of the framework. The robot was able to complete the tasks despite multiple contact state changes and errors encountered, increasing the success rate averaged across the tasks from 70.0% to 95.3%.

author = {Austin S. Wang and Oliver Kroemer},
title = {Learning Robust Manipulation Strategies with Multimodal State Transition Models and Recovery Heuristics},
booktitle = {International Conference on Robotics and Automation (ICRA)},
year = {2019},
month = {January},
} 2019-03-18T09:21:14-04:00