Bounce and Learn: Modeling Scene Dynamics with Real-World Bounces

Senthil Purushwalkam, Abhinav Gupta, Danny Kaufman and Bryan Russell
Conference Paper, Seventh International Conference on Learning Representations (ICLR 2019), May, 2019

Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.


We introduce an approach to model surface properties governing bounces in everyday scenes. Our model learns end-to-end, starting from sensor inputs, to predict post-bounce trajectories and infer two underlying physical properties that govern bouncing – restitution and effective collision normals. Our model, Bounce and Learn, comprises two modules — a Physics Inference Module (PIM) and a Visual Inference Module (VIM). VIM learns to infer physical parameters for locations in a scene given a single still image, while PIM learns to model physical interactions for the prediction task given physical parameters and observed pre-collision 3D trajectories. To achieve our results, we introduce the Bounce Dataset comprising 5K RGB-D videos of bouncing trajectories of a foam ball to probe surfaces of varying shapes and materials in everyday scenes including homes and offices. Our proposed model learns from our collected dataset of real-world bounces and is bootstrapped with additional information from simple physics simulations. We show on our newly collected dataset that our model out-performs baselines, including trajectory fitting with Newtonian physics, in predicting post-bounce trajectories and inferring physical properties of a scene.

author = {Senthil Purushwalkam and Abhinav Gupta and Danny Kaufman and Bryan Russell},
title = {Bounce and Learn: Modeling Scene Dynamics with Real-World Bounces},
booktitle = {Seventh International Conference on Learning Representations (ICLR 2019)},
year = {2019},
month = {May},
} 2019-05-24T13:54:06-04:00