Carnegie Mellon University
Lucas-Kanade 20 Years On: A Unifying Framework

Simon Baker and Iain Matthews
International Journal of Computer Vision, Vol. 56, No. 3, pp. 221 - 255, March, 2004.

  • Adobe portable document format (pdf) (463KB)
Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.

Since the Lucas-Kanade algorithm was proposed in 1981 image alignment has become one of the most widely used techniques in computer vision. Applications range from optical flow and tracking to layered motion, mosaic construction, and face coding. Numerous algorithms have been proposed and a wide variety of extensions have been made to the original formulation. We present an overview of image alignment, describing most of the algorithms and their extensions in a consistent framework. We concentrate on the inverse compositional algorithm, an efficient algorithm that we recently proposed. We examine which of the extensions to Lucas-Kanade can be used with the inverse compositional algorithm without any significant loss of efficiency, and which cannot. In this paper, Part 1 in a series of papers, we cover the quantity approximated, the warp update rule, and the gradient descent approximation. In future papers we will cover the choice of the norm, how to allow linear appearance variation, how to impose priors on the parameters, and various techniques to avoid local minima.

Image alignment, Lucas-Kanade, a unifying framework, additive vs. compositional algorithms, forwards vs. inverse algorithms, the inverse compositional algorithm, efficiency,steepest descent, Gauss-Newton, Newton, Levenberg-Marquardt

Sponsor: US Office of Naval Research
Grant ID: N00014-00-1-0915
Associated Center(s) / Consortia: Vision and Autonomous Systems Center
Associated Lab(s) / Group(s): Vision for Safe Driving, People Image Analysis Consortium, Face Group
Associated Project(s): AAM Fitting Algorithms, Lucas-Kanade 20 Years On, Face Model Building and Fitting, Face and Facial Feature Tracking
Number of pages: 54

Text Reference
Simon Baker and Iain Matthews, "Lucas-Kanade 20 Years On: A Unifying Framework," International Journal of Computer Vision, Vol. 56, No. 3, pp. 221 - 255, March, 2004.

BibTeX Reference
   author = "Simon Baker and Iain Matthews",
   title = "Lucas-Kanade 20 Years On: A Unifying Framework",
   journal = "International Journal of Computer Vision",
   pages = "221 - 255",
   month = "March",
   year = "2004",
   volume = "56",
   number = "3",