Integrating Perception and Planning for Humanoid Autonomy

Abstract

Today's agile humanoid robots are testament to the impressive advances in the design of biped mechanisms and control in recent robotics history. The big challenge, however, remains to properly exploit the generality and flexibility of humanoid platforms during fully autonomous operation in obstacle-filled and dynamically changing environments. Increasing effort has thus been focused on the challenges arising for perception and motion planning, as well as the interplay between both, as foundations of humanoid autonomy. This thesis explores appropriate approaches to perception on humanoids and ways of coupling sensing and planning to generate navigation and manipulation strategies that can be executed reliably. We investigate perception methods employing on- and off-body sensors that are combined with an efficient motion planner to allow the humanoid robot HRP-2 and Honda's ASIMO to traverse complex and unpredictably changing environments. We examine how predictive information about the future state of the world gathered from observation enables navigation in the presence of challenging moving obstacles. We show how programmable graphics hardware can be exploited to create a novel, model-based 3D tracking system able to robustly address the difficulties of real-time sensing specifically encountered on a locomoting humanoid. This thesis argues furthermore that reliability of autonomous operation can be improved by reasoning about perception during the planning process, rather than maintaining the traditional separation of the sensing and planning stages. We use the humanoid robots ARMAR-III and HRP-2 to investigate and validate such planning for perceptive capability in manipulation and navigation scenarios. While humanoid robots serve as the motivating challenge and application domain for this thesis, much of the resulting work is general in nature and has applications in other areas of robotics and computer vision.