RI | Research | Projects | Light-fields

Text only version of this site

Light-fields This project is no longer active. Head: Simon Baker Contact: Simon Baker Mailing address: Carnegie Mellon University Robotics Institute 5000 Forbes Avenue Pittsburgh, PA 15213 Associated center: VASC

Jump to: Project Description | Personnel | Publications

Project Description

Although the concept of the "light-field" of a scene dates back far earlier, Levoy and Hanrahan's seminal 1996 SIGGRAPH "Light-Field Rendering" paper created a large amount of interest in the graphics community in the capture and use of the light-field for rendering. The light-field is also of great interest in the computer vision community where it has a variety of other applications besides rendering. We have explored a couple of such applications as outlined below:

Stereo

When is stereo unique and when is it inherently ambiguous? We have investigated this theoretical question and derived a concise characterization of when the stereo problem (given the entire light-field) has a unique solution and when there are multiple scenes that could have generated the same set of photometric measurements (the light-field.)

Face Recognition

We have developed an appearance-based face recognition algorithm that can operate given any subset of the light-field of the face. We call this algorithm "eigen light-fields" because it is a generalization of "eigen faces." The training and testing subsets of the light-field do not need to overlap. Hence, our algorithm can perform face recognition across pose; for example, it can recognize a person from a profile view even though the algorithm has only seen that person from the front in the training data.

Past members

	Name	Title	Email Address
	Simon Baker	Adjunct Faculty (Adjunct)
	Ralph Gross	PhD Student, ISRI	rgross@cs.cmu.edu
	Takeo Kanade	U.A. and Helen Whitaker University Prof.	tk@cs.cmu.edu
	Iain Matthews	Senior Systems Scientist	iainm@cs.cmu.edu (Currently On Leave)

Publications

Note: This list may not be comprehensive. It contains only those publications in the RI publications database. Entries are listed in reverse chronological order.

• Appearance-Based Face Recognition and Light-Fields
  R. Gross, I. Matthews, and S. Baker
  IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 26, No. 4, April, 2004, pp. 449 - 465. [Abstract]
  Download: pdf [775 KB] copyrighted

• When is the Shape of a Scene Unique Given its Light-Field: A Fundamental Theorem of 3D Vision?
  S. Baker, T. Sim, and T. Kanade
  IEEE Transaction on Pattern Analysis and Machine Intelligence, Vol. 25, No. 1, January, 2003, pp. 100 - 109. [Abstract]
  Download: pdf [155 KB] copyrighted

• Fisher Light-Fields for Face Recognition Across Pose and Illumination
  R. Gross, I. Matthews, and S. Baker
  Proceedings of the German Symposium on Pattern Recognition (DAGM), September, 2002. [Abstract]
  Download: pdf [258 KB], ps.gz [3085 KB] copyrighted

• Appearance Based Face Recognition and Light-Fields
  R. Gross, I. Matthews, and S. Baker
  tech. report CMU-RI-TR-02-20, Robotics Institute, Carnegie Mellon University, August, 2002. [Abstract]
  Download: pdf [5701 KB] copyrighted

• Eigen Light-Fields and Face Recognition Across Pose
  R. Gross, I. Matthews, and S. Baker
  Proceedings of the IEEE International Conference on Automatic Face and Gesture Recognition, May, 2002, pp. 1 - 7. [Abstract]
  Download: pdf [690 KB], ps.gz [3335 KB] copyrighted

• A Characterization of Inherent Stereo Ambiguities
  S. Baker, T. Sim, and T. Kanade
  Proceedings of the 8th IEEE International Conference on Computer Vision (ICCV'01), Vancouver, British Columbia, July, 2001. [Abstract]
  Download: pdf [107 KB], ps.gz [67 KB] copyrighted

• A Fundamental Theorem of Stereo?
  S. Baker, T. Sim, and T. Kanade
  tech. report CMU-RI-TR-01-07, Robotics Institute, Carnegie Mellon University, June, 2001. [Abstract]
  Download: pdf [116 KB], ps.gz [75 KB] copyrighted