Generalizable Neural Dynamics Modeling for Complex Deformable Object Manipulation - Robotics Institute Carnegie Mellon University
Loading Events

MSR Thesis Presentation

May

21
Thu
Chengyang Zhao MSR Student Robotics Institute,
Carnegie Mellon University
Thursday, May 21
12:30 pm to 1:30 pm
Generalizable Neural Dynamics Modeling for Complex Deformable Object Manipulation
Abstract:
Hair care is an essential daily activity, yet it remains inaccessible to individuals with limited mobility and challenging for autonomous robot systems due to the fine-grained physical structure and complex dynamics of hair.
This thesis presents DYMO-Hair, a model-based robot hair care system. We introduce a novel dynamics learning paradigm that is suited for volumetric quantities such as hair, relying on an action-conditioned latent state editing mechanism, coupled with a compact 3D latent space of diverse hairstyles to improve generalizability. This latent space is pre-trained at scale using a novel hair physics simulator, enabling generalization across previously unseen hairstyles. Using the dynamics model with a Model Predictive Path Integral (MPPI) planner, DYMO-Hair is able to perform multimodal goal-conditioned hair styling, supporting both visual references and natural language instructions as goal configurations.
Experiments in simulation demonstrate that DYMO-Hair’s dynamics model outperforms baselines on capturing local deformation for diverse, unseen hairstyles. DYMO-Hair further outperforms baselines in closed-loop visual goal-conditioned hair styling tasks on unseen hairstyles, with an average of 22% lower final geometric error and 42% higher success rate than the state-of-the-art system. Real-world experiments exhibit zero-shot transferability of our system to wigs, achieving consistent success on challenging unseen hairstyles where the state-of-the-art system fails. We further conduct a user study to evaluate the system’s language goal-conditioned styling capability, suggesting improved goal achievement and progress consistency over zero-shot VLM-based methods.
Together, these results introduce a foundation for model-based robot hair care, advancing toward more generalizable, flexible, and accessible robot hair styling in unconstrained physical environments.
Committee:
Prof. Jean Oh (Chair)
Prof. Jeffrey Ichnowski
Prof. David Held
Uksang Yoo