Learning to Plan Precise and Task-oriented Grasps for Autonomous Robotic Assembly

Abstract

Robust, precise, and task-oriented grasp planning is vital for autonomous robotic assembly. It requires reasoning about the object geometry and preconditions of a task so as to properly grasp an object and complete the down-stream tasks. However, achieving such grasps is challenging due to the difficulty in understanding constraints and dynamics during objects interaction, as well as noise in control and unknown object properties. To tackle this problem, we proposed two data-driven, learning-based frameworks to plan precise and task-oriented grasps.

Our first experiment focuses on planning robotic grasps that are both robust and precise by training two convolutional neural networks - one to predict the robustness of a grasp and another to predict a distribution of post-grasp object displacements. Our networks are trained with depth images in simulation on a dataset of over 1000 industrial parts and were successfully deployed on a real robot without having to be further fine-tuned. The proposed displacement estimator achieves a mean prediction errors of 0.68cm and 3.42deg on novel objects in real world experiments.

Our second experiment further investigates whether a grasp is appropriate to a given downstream task. We propose a method that optimizes for grasp robustness, precision, and task performance all together by learning three cascaded networks. We collect training data based on large-scale self-supervised grasp simulation with procedurally generated objects. We form the training process as a curriculum learning problem, and perform both simulated and real world experiments on two common assembly tasks: inserting gears onto pegs and aligning brackets into corners. Our model achieves 4.28mm precision for bracket insertion and 1.44mm precision for gear insertion in real world experiments.