Carnegie Mellon University
Supervised Descent Method

Xuehan Xiong
doctoral dissertation, tech. report CMU-RI-TR-15-28, Robotics Institute, Carnegie Mellon University, September, 2015

  • Adobe portable document format (pdf) (35MB)
Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.

In this dissertation, we focus on solving Nonlinear Least Squares problems using a supervised approach. In particular, we developed a Supervised Descent Method (SDM), performed thorough theoretical analysis, and demonstrated its effectiveness on optimizing analytic functions, and four other real-world applications: Inverse Kinematics, Rigid Tracking, Face Alignment (frontal and multi-view), and 3D Object Pose Estimation. In Rigid Tracking, SDM was able to take advantage of more robust features, such as, HoG and SIFT. Those non-differentiable image features were out of consideration of previous work because they relied on gradient-based methods for optimization. In Inverse Kinematics where we minimize a non-convex function, SDM achieved significantly better convergence than gradient-based approaches. In Face Alignment, SDM achieved state-of-the-arts results. Moreover, it was extremely computationally efficient, which makes it applicable for many mobile applications. In addition, we provided a unified view of several popular methods including SDM on sequential prediction, and reformulated them as a sequence of function compositions. Finally, we suggested some future research directions on SDM and sequential prediction.

nonlinear optimization, global optimization, non-convex optimization, nonlinear least squares, face alignment, facial feature tracking, pose estimation, inverse kinematics, imitation learning, learning from demonstration, policy derivation, neural network, stacking, gradient boosting, sequential prediction


Text Reference
Xuehan Xiong, "Supervised Descent Method," doctoral dissertation, tech. report CMU-RI-TR-15-28, Robotics Institute, Carnegie Mellon University, September, 2015

BibTeX Reference
   author = "Xuehan Xiong",
   title = "Supervised Descent Method",
   booktitle = "",
   school = "Robotics Institute, Carnegie Mellon University",
   month = "September",
   year = "2015",
   number= "CMU-RI-TR-15-28",
   address= "Pittsburgh, PA",