Abstract: Robots that manipulate crops must contend with plants that occlude themselves, deform under contact, and make the manipulator contact structures it neither targets nor sees in advance. This thesis argues that autonomous manipulation of crops in unstructured agricultural environments is best advanced not by choosing between model-based and learned approaches, but by integrating them where each is strongest: model-based components provide explicit geometric priors and safety constraints for contact-rich interaction, while learned components capture plant dynamics and visuomotor behaviors that resist hand-engineering.

Four technical contributions develop this argument. The first is a model-based skeletonization method that recovers tree branch structure from RGB-D views using an occlusion-likelihood field and a min-cost path search to reason explicitly about unobserved geometry. The second is a graph neural network trained on mass-spring-damper simulations that predicts how a tree deforms under push. The third is an end-to-end diffusion policy trained on demonstrations collected with a handheld shear-gripper that drives pepper harvesting in an unprotected outdoor field. The fourth is a reactive safety layer that wraps a learned visuomotor policy with a control barrier function quadratic program whose constraints are drawn from an occupancy map updated online from contact estimates, enabling safe reaching through cluttered canopies.

Taken together, the four contributions trace a path from offline geometric reasoning about static plants to safe contact-rich interaction in clutter, showing that robust agricultural manipulation emerges not from end-to-end learning alone, but from a synthesis in which model-based safety enables reliable learned interaction.

Committee:
George Kantor (chair)
Oliver Kroemer
Dave Held
Maren Bennewitz (University of Bonn)