Leveraging learning techniques for agricultural robotics

Abstract

Effective plant breeding requires scientists to find correspondences between genetic markers and desirable physical traits of the genotype. While the cost for gene sequencing has gone down by several orders of magnitude in the last two decades, measuring physical traits of a plant at scale is still a labour intensive task making it borderline intractable. This creates an opportunity for robotics to fill this gap in order to accelerate the breeding pipeline. Aided by current progress in machine learning, in this thesis we elucidate methods to perform high throughput non contact and contact based plant phenotyping. We developed a general purpose computer vision pipeline for plant physical trait detection and semantic segmentation. We tested this pipeline for Sorghum stalk counting and stalk width measurement from images. We discuss the shortcomings of having separate perception and manipulation modules for contact based phenotyping and propose an end to end reinforcement learning framework for learning from RGBD observations. We propose a principled online learning approach to weight different auxiliary losses in order to accelerate reinforcement learning. Our approach gives a 3x speed up for reaching and manipulation tasks in three simulated environments.