Abstract:
Humans convey their thoughts, emotions, and intentions through a concert of social displays: voice, facial expressions, hand gestures, and body posture. Despite advances in machine perception technology, machines are unable to discern the subtle and momentary nuances that carry so much of the information and context of human communication. The encoding of conveyed information by human body movements is still poorly understood, and a major obstacle to scientific progress in understanding human behavior is the inability to measure the full spectrum of social signals in groups of interacting individuals.

In this talk, I will describe my early exploration in building sensors that can capture the full spectrum of human social signaling—from voice, to facial expressions, to hand gestures, to body posture—among groups of multiple people. Leveraging more than 500 synchronized cameras, our method enables us to markerlessly measure subtle 3D movements of interacting people, providing a new opportunity to computationally study social interaction. I will also talk about my ongoing efforts to understand social interaction in a predictive way, based on our novel dataset containing 3D social signals from hundreds of participants.

Bio:
Hanbyul Joo is a Ph.D. candidate in the Robotics Institute, Carnegie Mellon University. His research focuses on measuring social signals in interpersonal social communication to computationally model social behavior, using tools of computer vision, computer graphics, and machine learning. Hanbyul has been developing the Panoptic Studio at CMU, a sensing system designed to capture social interaction using more than 500 synchronized cameras. Hanbyul’s research has been covered in various media outlets including Discovery, Reuters, IEEE Spectrum, NBC News, Voice of America, The Verge, and WIRED. He is a recipient of the Samsung Scholarship.

Areas of Interest: