Robust Tracking and Structure from Motion with Sampling Method

Abstract

Robust tracking and structure from motion (SFM) are fundamental problems in computer vision that have important applications for robot visual navigation and other computer vision tasks. Although the geometry of the SFM problem is well understood and effective optimization algorithms have been proposed, SFM is still difficult to apply in practice. The reason is twofold. First, finding correspondences, or "data association", is still a challenging problem in practice. For visual navigation tasks, the correspondences are usually found by tracking, which often assumes constancy in feature appearance and smoothness in camera motion so that the search space for correspondences is much reduced. Therefore tracking itself is intrinsically difficult under degenerate conditions, such as occlusions, or abrupt camera motion which violates the assumptions for tracking to start with. Second, the result of SFM is often observed to be extremely sensitive to the error in correspondences, which is often caused by the failure of the tracking. This thesis aims to tackle both problems simultaneously. We attribute the difficulty of applying SFM in practice to the failure of tracking algorithms under those degenerate conditions often seen in the uncontrolled environments. I propose to integrate the SFM with tracking so that the tracking algorithms can have the structure information recovered from SFM together with the image data so that it can explicitly reason about occlusions, ambiguous scenes and abrupt camera motion. Therefore it becomes more robust against those degenerate conditions by properly detecting them. In addition, the SFM results become more reliable by reducing error in the correspondences found the tracking algorithms. I aim to achieve this in a probabilistically sound way. Representing the uncertainty is one of the most important issues for a probabilistic framework can be applied. I propose a sampling method to capture the uncertainty in both tracking and SFM. The uncertainty is naturally represented with sample sets. A probabilistic filtering algorithm is developed to propagate the uncertainty through time. With the sample-based representation, our system can capture the uncertainty in both tracking and SFM under degenerate conditions, therefore exhibiting improved robustness by taking proper measures against them, which are usually difficult for the traditional approaches to achieve. We believe that with this sampling-based probabilistic framework, we are one step closer to a SFM system that can perform reliably in real robot navigation tasks.