Abstract:

Despite significant recent progress in computer vision and machine learning, personalized autonomous agents often still don’t participate robustly and safely across tasks in our environment. We think this is largely because they lack an ability to anticipate, which in turn is due to a missing understanding about what is happening behind the seen, i.e., in occluded or unobserved parts of an image. To develop this technology to anticipate, we think answers to four foundational questions are needed: (1) How can methods accurately forecast high-dimensional observations?; (2) How can algorithms holistically understand objects, e.g., when reasoning about occluded parts?; (3) How can accurate probabilistic models be recovered from limited amounts of labeled data and for rare events?; and (4) How can autonomous agents be trained effectively to collaborate?

In this talk we present vignettes of our research to address those questions. We discuss panoptic forecasting, a new task to study algorithms for high-dimensional forecasting. We then illustrate methods for holistic object understanding, addressing tasks like semantic a-modal instance-level video object segmentation (SAIL-VOS) and its 3D counterpart. Time permitting, we sketch recent advances to train collaborative embodied agents.

Bio:

Alex Schwing is an Assistant Professor at the University of Illinois at Urbana-Champaign working with talented students on computer vision and machine learning topics. He received his B.S. and diploma in Electrical Engineering and Information Technology from Technical University of Munich in 2006 and 2008 respectively, and obtained a PhD in Computer Science from ETH Zurich in 2014. Afterwards he joined University of Toronto as a postdoctoral fellow until 2016. His research interests are in the area of computer vision and machine learning, where he has co-authored numerous papers on topics in scene understanding, inference and learning algorithms, deep learning, image and language processing and generative modeling. His PhD thesis was awarded an ETH medal and his team’s research was awarded an NSF CAREER award. For additional info, please browse to http://alexander-schwing.de.

Homepage:  http://alexander-schwing.de

Sponsored in part by:   Facebook Reality Labs Pittsburgh