DexPilot: Vision-Based Teleoperation of Dexterous Robotic Hand-Arm System
Abstract
Teleoperation offers the possibility of imparting robotic systems with sophisticated reasoning skills, intuition, and creativity to perform tasks. However, teleoperation solutions for high degree-of-actuation (DoA), multi-fingered robots are generally cost-prohibitive, while low-cost offerings usually offer reduced degrees of control. Herein, a low-cost, depth-based teleoperation system, DexPilot, was developed that allows for complete control over the full 23 DoA robotic system by merely observing the bare human hand. DexPilot enabled operators to solve a variety of complex manipulation tasks that go beyond simple pick-and-place operations and performance was measured through speed and reliability metrics. DexPilot cost-effectively enables the production of high dimensional, multi-modality, state-action data that can be leveraged in the future to learn sensorimotor policies for challenging manipulation tasks. The videos of the experiments can be found at:
https://sites.google.com/view/dex-pilot.
BibTeX
@conference{Handa-2020-126758,author = {Ankur Handa and Karl Van Wyk and Wei Yang and Jacky Liang and Yu-Wei Chao and Qian Wan and Stan Birchfield and Nathan D. Ratliff and Dieter Fox},
title = {DexPilot: Vision-Based Teleoperation of Dexterous Robotic Hand-Arm System},
booktitle = {Proceedings of (ICRA) International Conference on Robotics and Automation},
year = {2020},
month = {May},
pages = {9164 - 9170},
}