Abstract:   Equipping AI and robotic systems with the ability to understand human behavior is essential for enabling them to assist people across a wide range of everyday applications. This need is more pressing than ever: the heaviest consumers of such knowledge are no longer perception systems alone, but robot policies that must learn to act in the physical world. Yet the high-quality 3D human motion data required to learn this knowledge remains extremely scarce.

In this talk, I will present our lab’s efforts to scale and enrich 3D human motion data by capturing everyday movements and natural human-object interactions, with the ultimate goal of teaching robots to move like humans.

I will first introduce our multi-year effort in building multi-camera capture systems, from Panoptic Studio to ParaHome, and most recently OmniRoboHome, a new system designed to capture human-object interactions in natural home environments. Next, I will present a complementary direction: learning everyday interactions and affordances from generative image and video models, which offer a scalable source of human behavior priors that capture systems alone cannot reach. Finally, I will discuss the missing pieces that vision and image models cannot provide, including physics, contact, and the gap across diverse embodiments, together with our recent efforts to fill them.

Bio:  Hanbyul Joo is an associate professor at Seoul National University (SNU) in the Department of Computer Science and Engineering. Before joining SNU, Hanbyul was a Research Scientist at Facebook AI Research (FAIR), Menlo Park. Hanbyul received his PhD from the Robotics Institute at Carnegie Mellon University. He is a recipient of the Samsung Scholarship, the Okawa Foundation Research Grant, and the Best Student Paper Award at CVPR 2018.

Homepage: https://jhugestar.github.io/

Sponsor:
The VASC seminar is generously sponsored by HeyGen, an all-in-one
AI-powered video generation platform that leverages advances in
computer vision, generative modeling, and multimodal learning to make
high-quality video creation both scalable and accessible.