Generalizable Neural Dynamics Modeling for Complex Deformable Object Manipulation
Abstract:
Hair care is an essential daily activity, yet it remains inaccessible to individuals with limited mobility and challenging for autonomous robot systems due to the fine-grained physical structure and complex dynamics of hair.
This thesis presents DYMO-Hair, a model-based robot hair care system. We introduce a novel dynamics learning paradigm that is suited for volumetric quantities such as hair, relying on an action-conditioned latent state editing mechanism, coupled with a compact 3D latent space of diverse hairstyles to improve generalizability. This latent space is pre-trained at scale using a novel hair physics simulator, enabling generalization across previously unseen hairstyles. Using the dynamics model with a Model Predictive Path Integral (MPPI) planner, DYMO-Hair is able to perform multimodal goal-conditioned hair styling, supporting both visual references and natural language instructions as goal configurations.
Experiments in simulation demonstrate that DYMO-Hair’s dynamics model outperforms baselines on capturing local deformation for diverse, unseen hairstyles. DYMO-Hair further outperforms baselines in closed-loop visual goal-conditioned hair styling tasks on unseen hairstyles, with an average of 22% lower final geometric error and 42% higher success rate than the state-of-the-art system. Real-world experiments exhibit zero-shot transferability of our system to wigs, achieving consistent success on challenging unseen hairstyles where the state-of-the-art system fails. We further conduct a user study to evaluate the system’s language goal-conditioned styling capability, suggesting improved goal achievement and progress consistency over zero-shot VLM-based methods.
Together, these results introduce a foundation for model-based robot hair care, advancing toward more generalizable, flexible, and accessible robot hair styling in unconstrained physical environments.
Hair care is an essential daily activity, yet it remains inaccessible to individuals with limited mobility and challenging for autonomous robot systems due to the fine-grained physical structure and complex dynamics of hair.
This thesis presents DYMO-Hair, a model-based robot hair care system. We introduce a novel dynamics learning paradigm that is suited for volumetric quantities such as hair, relying on an action-conditioned latent state editing mechanism, coupled with a compact 3D latent space of diverse hairstyles to improve generalizability. This latent space is pre-trained at scale using a novel hair physics simulator, enabling generalization across previously unseen hairstyles. Using the dynamics model with a Model Predictive Path Integral (MPPI) planner, DYMO-Hair is able to perform multimodal goal-conditioned hair styling, supporting both visual references and natural language instructions as goal configurations.
Experiments in simulation demonstrate that DYMO-Hair’s dynamics model outperforms baselines on capturing local deformation for diverse, unseen hairstyles. DYMO-Hair further outperforms baselines in closed-loop visual goal-conditioned hair styling tasks on unseen hairstyles, with an average of 22% lower final geometric error and 42% higher success rate than the state-of-the-art system. Real-world experiments exhibit zero-shot transferability of our system to wigs, achieving consistent success on challenging unseen hairstyles where the state-of-the-art system fails. We further conduct a user study to evaluate the system’s language goal-conditioned styling capability, suggesting improved goal achievement and progress consistency over zero-shot VLM-based methods.
Together, these results introduce a foundation for model-based robot hair care, advancing toward more generalizable, flexible, and accessible robot hair styling in unconstrained physical environments.
Committee:
Prof. Jean Oh (Chair)
Prof. Jeffrey Ichnowski
Prof. David Held
Uksang Yoo
Prof. Jean Oh (Chair)
Prof. Jeffrey Ichnowski
Prof. David Held
Uksang Yoo
