Simulation-guided Design for Vision-based Tactile Sensing on a Soft Robot Gripper

Abstract

Soft pneumatic robot manipulators have garnered widespread interest due to their compliance and flexibility, which enable soft, non-destructive grasping and strong adaptability to complex working environments. Tactile sensing is crucial for these manipulators to provide real-time contact information for control and manipulation, but most tactile sensors are either inflexible, low-resolution, expensive, or hard to manufacture.

Vision-based tactile sensing has been used in soft pneumatic robot manipulators to achieve a high spatial resolution of contact information. However, designing a high-quality vision-based tactile sensor for soft pneumatic robot manipulators is challenging for two reasons. Achieving steady high-fidelity sensing signals on a deformable robot is non-trivial, and manufacturing time and expense make design iteration in the real world unpractical.

In this thesis, we present a physics-based optical simulation pipeline to guide the design of vision-based tactile sensing on a soft robot finger. Our simulation pipeline enables faster iteration cycles and automatic optimization of design parameters, eliminating the need to manufacture the entire robot finger for each design iteration. The simulation utilizes physics-based rendering (PBR) with highly accurate optical modeling of the robot finger to ensure that performance improvements in tactile sensing in the simulation are transferable to the real-world robot finger. We introduce a fast numerical metric to test tactile sensing performance at different robot states and contact locations to evaluate designs. During the optimization, we perform a grid search on possible combinations of light color choices for optical fibers based on defined metrics. Then, we apply the covariance matrix adaptation evolution strategy (CMA-ES) as a numerical optimization method to iteratively fine-tune the directions of the optical fibers. We compare the tactile sensing performance of the optimized design with the baseline design and demonstrate an improvement.

This thesis makes the following contributions: a) a ready-to-use simulation pipeline for any vision-based tactile sensor; b) providing a new design paradigm that uses optical simulation as a testing and optimization platform; c) a new metric for evaluating the sensing performance of vision-based tactile sensors.