Abstract:
Robots are becoming more versatile, increasing the available opportunities to use them in situations that aid people in everyday tasks. For example, recent research has investigated robot manipulators for assisting people with motor impairments in activities of daily living such as eating a meal. To form successful collaborations in these interactions, researchers need to design robots capable of understanding and acting on human intentions. While intentions can be directly expressed in certain situations, (e.g. “Hey Siri, tell me what time it is.”), it can be burdensome in others, such as when people have relatively long-term, continuous interactions with a robot. Fortunately, humans continuously display a rich variety of nonverbal behaviors that indirectly signal their intentions, without added burden or prompting. One such behavior is eye gaze, which previous research has shown to be linked to intent. In this talk, I will discuss the creation of a new dataset, Human And Robot Multimodal Observations of Natural Interactive Collaboration (HARMONIC). This dataset provides researchers with an opportunity to study people’s nonverbal behaviors as they co-manipulate an assistive robot in an eating task. The data consist of 24 participants each controlling a 6 degree of freedom robot by using a 2 degree of freedom joystick. The participant and robot collaborate on the eating task using shared control at four different levels of assistance provided by the robot. The dataset includes time synchronized ego-centric video, eye gaze data, and joystick action. I will then discuss the first steps we are taking to understand hand eye coordination in this task, which will give insight into how to incorporate multiple nonverbal signals for improved intent recognition and robotic assistance.

Committee:
Henny Admoni (advisor)
Kris Kitani (advisor)
Louis-Philippe Morency
Xiang Zhi Tan