Learning to Switch between Sensorimotor Primitives using Multimodal Haptic Signals

Zhe Su, Oliver Kroemer, Gerald E. Loeb, Gaurav S. Sukhatme and Stefan Schaal
Conference Paper, International Conference on the Simulation of Adaptive Behavior (SAB), January, 2016

Download Publication

Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.


Most manipulation tasks can be decomposed into sequences of sensorimotor primitives. These primitives often end with characteristic sensory events, e.g., making or breaking contact, which indicate when the sensorimotor goal has been reached. In this manner, the robot can monitor the tactile signals to determine when to switch between primitives. In this paper, we present a framework for automatically segmenting contact-based manipulation tasks into sequences of sensorimotor primitives using multimodal haptic signals. These signals include both the robot’s end-effector position as well as the low- and high-frequency components of its tactile sensors. The resulting segmentation is used to learn to detect when the robot has reached a sensorimotor goal and it should therefore switch to the next primitive. The proposed framework was evaluated on guided peg-in-hole tasks. The experiments show that the framework can extract the subtasks of the manipulations and the sensorimotor goals can be accurately detected.

author = {Zhe Su and Oliver Kroemer and Gerald E. Loeb and Gaurav S. Sukhatme and Stefan Schaal},
title = {Learning to Switch between Sensorimotor Primitives using Multimodal Haptic Signals},
booktitle = {International Conference on the Simulation of Adaptive Behavior (SAB)},
year = {2016},
month = {January},
} 2019-03-12T13:19:49-04:00