Learning to Track at 100 FPS with Deep Regression Networks

David Held, Sebastian Thrun and Silvio Savarese
Conference Paper, European Conference on Computer Vision (ECCV), October, 2017

Download Publication

Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.


Machine learning techniques are often used in computer vision due to their ability to leverage large amounts of training data to improve performance. Unfortunately, most generic object trackers are still trained from scratch online and do not benefit from the large number of videos that are readily available for offline training. We propose a method for using neural networks to track generic objects in a way that allows them to improve performance by training on labeled videos. Previous attempts to use neural networks for tracking are very slow to run and not practical for real-time applications. In contrast, our tracker uses a simple feed-forward network with no online training required, allowing our tracker to run at 100 fps during test time. Our tracker trains from both labeled video as well as a large collection of images, which helps prevent overfitting. The tracker learns generic object motion and can be used to track novel objects that do not appear in the training set. We test our network on a standard tracking benchmark to demonstrate our tracker’s state-of-the-art performance. Our network learns to track generic objects in real-time as they move throughout the world.

author = {David Held and Sebastian Thrun and Silvio Savarese},
title = {Learning to Track at 100 FPS with Deep Regression Networks},
booktitle = {European Conference on Computer Vision (ECCV)},
year = {2017},
month = {October},
publisher = {Springer},
} 2018-02-07T13:26:17-04:00