Graphics enhanced version of this site

Light-fields
This project is no longer active.

Head: Simon Baker
Contact: Simon Baker

Mailing address:
Carnegie Mellon University
Robotics Institute
5000 Forbes Avenue
Pittsburgh, PA 15213

Associated center: VASC

Although the concept of the "light-field" of a scene dates back far earlier, Levoy and Hanrahan's seminal 1996 SIGGRAPH "Light-Field Rendering" paper created a large amount of interest in the graphics community in the capture and use of the light-field for rendering. The light-field is also of great interest in the computer vision community where it has a variety of other applications besides rendering. We have explored a couple of such applications as outlined below:

Stereo

When is stereo unique and when is it inherently ambiguous? We have investigated this theoretical question and derived a concise characterization of when the stereo problem (given the entire light-field) has a unique solution and when there are multiple scenes that could have generated the same set of photometric measurements (the light-field.)

Face Recognition

We have developed an appearance-based face recognition algorithm that can operate given any subset of the light-field of the face. We call this algorithm "eigen light-fields" because it is a generalization of "eigen faces." The training and testing subsets of the light-field do not need to overlap. Hence, our algorithm can perform face recognition across pose; for example, it can recognize a person from a profile view even though the algorithm has only seen that person from the front in the training data.

Name - Title <Email Address>
• Simon Baker - Adjunct Faculty (Adjunct)
• [Home] Ralph Gross - PhD Student, ISRI <rgross@cs.cmu.edu>
• [Home] Takeo Kanade - U.A. and Helen Whitaker University Prof., RI/CS <tk@cs.cmu.edu>
• [Home] Iain Matthews - Senior Systems Scientist <iainm@cs.cmu.edu (Currently On Leave)>

Note: This list may not be comprehensive. It contains only those publications in the RI publications database. Entries are listed in reverse chronological order.

• Appearance-Based Face Recognition and Light-Fields
  R. Gross, I. Matthews, and S. Baker
  IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 26, No. 4, April, 2004, pp. 449 - 465. [Abstract]
  Download: pdf [775 KB] copyrighted

• When is the Shape of a Scene Unique Given its Light-Field: A Fundamental Theorem of 3D Vision?
  S. Baker, T. Sim, and T. Kanade
  IEEE Transaction on Pattern Analysis and Machine Intelligence, Vol. 25, No. 1, January, 2003, pp. 100 - 109. [Abstract]
  Download: pdf [155 KB] copyrighted

• Fisher Light-Fields for Face Recognition Across Pose and Illumination
  R. Gross, I. Matthews, and S. Baker
  Proceedings of the German Symposium on Pattern Recognition (DAGM), September, 2002. [Abstract]
  Download: pdf [258 KB], ps.gz [3085 KB] copyrighted

• Appearance Based Face Recognition and Light-Fields
  R. Gross, I. Matthews, and S. Baker
  tech. report CMU-RI-TR-02-20, Robotics Institute, Carnegie Mellon University, August, 2002. [Abstract]
  Download: pdf [5701 KB] copyrighted

• Eigen Light-Fields and Face Recognition Across Pose
  R. Gross, I. Matthews, and S. Baker
  Proceedings of the IEEE International Conference on Automatic Face and Gesture Recognition, May, 2002, pp. 1 - 7. [Abstract]
  Download: pdf [690 KB], ps.gz [3335 KB] copyrighted

• A Characterization of Inherent Stereo Ambiguities
  S. Baker, T. Sim, and T. Kanade
  Proceedings of the 8th IEEE International Conference on Computer Vision (ICCV'01), Vancouver, British Columbia, July, 2001. [Abstract]
  Download: pdf [107 KB], ps.gz [67 KB] copyrighted

• A Fundamental Theorem of Stereo?
  S. Baker, T. Sim, and T. Kanade
  tech. report CMU-RI-TR-01-07, Robotics Institute, Carnegie Mellon University, June, 2001. [Abstract]
  Download: pdf [116 KB], ps.gz [75 KB] copyrighted