Abstract:

Humans routinely rely on the sense of touch to better perceive the world. In environments characterized by poor lighting, occlusions, limited fields of view, or sparse visual features, contact feedback often becomes a primary source of information for perceiving the environment and successfully completing manipulation tasks. Everyday examples include locating a light switch in the dark, retrieving an item from a high shelf, or reaching for a valve at the back of a kitchen sink cabinet where visual access is severely restricted. In such settings, humans actively reason about contacts to infer both the poses of objects of interest and the environmental obstacles in the workspace. Enabling robots to exhibit similar capabilities remains a fundamental challenge in autonomous manipulation. This thesis investigates search-based planning techniques that allow robots to effectively leverage contact feedback as a sensing modality, enabling robust manipulation under uncertainty.

This work focuses on two broad classes of manipulation problems in which contact plays a critical role. The first class concerns object pose uncertainty, where precise estimation of target object pose is required to complete high-precision manipulation tasks such as charger plug insertion, pipe assembly, or other tight-tolerance manipulation problems. In these settings, even small pose errors on the order of a few millimeters can lead to failure. This problem class studies how robots can actively use contacts during execution to reduce object pose uncertainty to a level sufficient for successful task completion. The second class of problems addresses manipulation under environmental uncertainty, where the locations and geometries of environmental obstacles are unknown or only partially observable. For example, when reaching into a cluttered kitchen sink cabinet with unknown obstacles and pipes, a robot must detect contacts, infer obstacle locations, and adapt its motion accordingly in order to safely reach the target (valve). Together, these two problem classes capture a wide range of real-world scenarios in which contact-driven reasoning is essential.

Planning under object pose uncertainty naturally falls within the framework of Partially Observable Markov Decision Processes (POMDPs), which are computationally expensive to solve, particularly in continuous and high-dimensional robotic domains. This thesis presents three complementary frameworks to address this challenge. The first is an experience-based preprocessing approach designed for semi-structured environments that require strong online performance. This framework leverages solutions to previously solved, similar POMDPs to accelerate future planning queries while maintaining theoretical guarantees on solution quality. An offline database of policies is constructed and queried at execution time based on the current problem instance, enabling fast online decision-making. The second framework targets less structured domains where preprocessing is impractical. It introduces an online closed-loop planning and execution approach that employs a hierarchical representation of uncertainty. By adaptively representing and reasoning about uncertainty, this method significantly reduces planning time, making online planning and execution feasible in more complex settings. The third contribution addresses a key computational bottleneck in these settings, namely the high cost of belief space transition computations. To mitigate this issue, the thesis proposes lazy heuristic search algorithms for POMDPs that defer expensive belief updates until they are necessary, using approximate Q-value estimators to guide search. These lazy solvers substantially reduce planning time while preserving solution quality.

For the problem class of manipulation under environmental uncertainty, this thesis develops an iterative planning and execution framework that tightly couples contact sensing, environment prediction, and motion planning. The system employs a torque-based contact detection and localization module capable of detecting contacts occurring anywhere along the robot manipulator. The history of detected contacts is used to construct a partial occupancy map of the workspace, which is then extrapolated using learned occupancy estimators. A motion planning module reasons over this estimated occupancy representation to compute actions that are likely to safely and efficiently move the robot toward the goal. The framework is evaluated in simulation and on a real UR10e manipulator across two challenging domestic tasks: manipulating a valve under a kitchen sink surrounded by pipes and retrieving a target object from a cluttered shelf.

Overall, this thesis takes a step toward manipulation in the blind, where robots explicitly leverage contact interactions as a primary sensing modality to plan and execute manipulation tasks under uncertainty. Rather than treating contact as a failure mode to be avoided, we treat it as an informative observation that can be actively exploited to reduce uncertainty and guide motion.

Thesis Committee:

Prof. Maxim Likhachev, Chair

Prof. Jeffrey Ichnowski

Prof. Oliver Kroemer

Prof. Mehmet Dogar, University of Leeds

A draft of the thesis document is available at: https://drive.google.com/drive/folders/1TCSil_tJksTWC_RweWGS2FHoZBjyWM8w?usp=drive_link