An Uncertain Future: Forecasting from Static Images using Variational Autoencoders - Robotics Institute Carnegie Mellon University

An Uncertain Future: Forecasting from Static Images using Variational Autoencoders

Conference Paper, Proceedings of (ECCV) European Conference on Computer Vision, pp. 835 - 851, October, 2016

Abstract

In a given scene, humans can easily predict a set of immediate future events that might happen. However, pixel-level anticipation in computer vision is difficult because machine learning struggles with the ambiguity in predicting the future. In this paper, we focus on predicting the dense trajectory of pixels in a scene---what will move in the scene, where it will travel, and how it will deform over the course of one second. We propose a conditional variational autoencoder as a solution to this problem. In this framework, direct inference from the image shapes the distribution of possible trajectories while latent variables encode information that is not available in the image. We show that our method predicts events in a variety of scenes and can produce multiple different predictions for an ambiguous future. We also find that our method learns a representation that is applicable to semantic vision tasks.

Notes
Associated Lab - Manipulation LabComputer Vision Lab

BibTeX

@conference{Walker-2016-103535,
author = {Jacob Walker and Carl Doersch and Abhinav Gupta and Martial Hebert},
title = {An Uncertain Future: Forecasting from Static Images using Variational Autoencoders},
booktitle = {Proceedings of (ECCV) European Conference on Computer Vision},
year = {2016},
month = {October},
pages = {835 - 851},
}