Abstract: As robots transition into unstructured human environments, achieving dexterous manipulation across diverse tasks becomes increasingly crucial yet remains challenging. This objective underscores the value of high-degree-of-freedom (DoF) robotic hands capable of versatile grasping and in-hand manipulation. Complementing the hardware, success in these settings also depends on intelligent and autonomous hand control strategies that provide the adaptability necessary for skill execution.

In this talk, I first introduce DeltaHands, a modular, non-anthropomorphic hand framework. Its design provides a reconfigurable space that enables rapid adaptation of the hand’s DoFs, mechanical properties, material composition, and sensor integration. While this non-anthropomorphic morphology offers mechanical simplicity, its divergence from the human hand complicates intuitive human-in-the-loop control and efficient skill acquisition. To bridge this gap, we explore human-to-robot motion mapping. Through user studies, we demonstrate that a kinematic-twin interface, which itself leverages the hand’s modularity, significantly reduces cognitive workload, and improves task success compared to vision-based motion retargeting. Using kinematic-twin interfaces, I will show how various dexterous skills can be acquired through learning from human demonstrations. Finally, I discuss the integration of multimodal tactile sensing into the DeltaHands framework to enable fine-grained manipulation. To exploit the transient yet information-rich nature of touch, I present an object representation learning method which extracts and preserves meaningful tactile signals during exploratory interactions to inform and guide downstream, object-centric manipulation policies. In conclusion, the concise and modular design of the non-anthropomorphic robotic hands, sensing systems, and control interfaces enables the direct transfer of human dexterity to robotic platforms and supports dexterous manipulation learning.

Committee:

Zeynep Temel (co-chair), Carnegie Mellon University

Oliver Kroemer (co-chair), Carnegie Mellon University

Nancy Pollard, Carnegie Mellon University

Oliver Brock, Technische Universität Berlin

Thesis Draft