Touch plays a critical role in how we perceive and interact with the environment in everyday life. Through touch, we can feel an object’s shape, sense how it moves, and manipulate it with greater dexterity and precision. Without touch, even simple tasks such as holding an object steadily or lighting a match become challenging. Despite recent advances in high-resolution tactile sensors that enable robots to capture rich contact information, accurately perceiving object properties, such as pose and shape, and leveraging tactile feedback for dexterous manipulation remain significant challenges. This thesis explores how tactile sensing can enable precise object perception and reliable dexterous manipulation.

To enable high-precision tactile perception, we introduce NormalFlow, a fast, accurate, and robust algorithm for tracking an object’s 6DoF pose using tactile sensing. By directly aligning tactile-derived surface normal maps instead of point clouds, NormalFlow consistently outperforms existing methods in tracking accuracy and robustness, and can even track low-texture objects such as ping-pong balls. It runs in 8 ms per frame on a CPU and maintains a rotational tracking error of only 2.5 degrees after a full 360-degree rotation. NormalFlow provides a foundation for future high-precision tactile perception and precise manipulation tasks.

Building on NormalFlow, we develop GelSLAM, a real-time tactile SLAM system for long-horizon object tracking and high-fidelity 3D reconstruction. By leveraging tactile-derived surface normals and curvatures for tracking and loop closure, GelSLAM effectively corrects accumulated drift and maintains global consistency. GelSLAM enables accurate tracking across tens of thousands of frames and hundreds of contact breaks, and is the first system to robustly reconstruct high-fidelity 3D object shapes with submillimeter accuracy using touch alone. GelSLAM extends tactile sensing beyond local contact measurements to global spatial understanding, opening new possibilities for tactile perception.

Beyond geometric perception, we investigate dynamic tactile sensing for liquid property estimation. Inspired by how humans shake a container to infer its contents, we develop a physics-inspired model of liquid oscillations and their corresponding tactile signals. By analyzing the frequency and decay of these oscillations, our method estimates liquid properties such as viscosity and height using limited training data. Experiments demonstrate accurate estimation of liquid properties, including subtle differences in sugar concentration. This work highlights the potential of dynamic tactile sensing to reveal physical properties that are difficult to observe through vision, extending tactile perception beyond surface geometry.

Beyond perceiving objects, we further explore how tactile feedback can improve dexterous manipulation. In the proposed work of this thesis, we introduce Contact Tamer, a learned contact-centric tactile policy that aims to make dexterous manipulation more efficient and reliable by directly regulating finger contacts. Unlike conventional object-centric policies that rely on visual feedback, Contact Tamer uses tactile sensing and proprioception to maintain desired contact behaviors in real time. Preliminary simulation results demonstrate improved task success and reduced unintended slipping. This work aims to enable robots to perform complex contact-rich manipulation tasks more robustly and efficiently.

Together, these works advance tactile sensing from understanding object geometry and physical properties to regulating contact during manipulation, laying the foundation for robots that can perceive and manipulate the physical world more precisely and reliably through touch.