The Robotics Institute
 Search the site
RI | People | Simon Baker

Text only version of this site

Simon Baker
Adjunct Faculty (Adjunct)

No longer a member of RI.

Jump to: Research interests | Keywords | Labs & Groups | Projects | Publications

Research interests

I do computer vision. Within this field, I am particularly interested in the following areas:

Faces:

Real-Time Non-Rigid Face Tracking / Active Appearance Model Fitting: 2D and 3D Face Model Building: Resolution Enhancement / Hallucinating Faces: Face Databases / PIE: Face Recognition Across Pose / Eigen Light-Fields: Gaze Estimation

3D Reconstruction and Vision/Graphics:

Fundamental Theorem of 3D Vision: Shape-From-Silhouette Across Time: Human Kinematic Modeling: Human Articulated Tracking / Markerless Motion Capture: Markerless Motion Transfer: Scene Flow: Spatio-Temporal View Interpolation: Textureless Layers

Vision Theory:

Fundamental Theorem of 3D Vision: Efficient Image Alignment / Lucas-Kanade 20 Years On Unifying Framework: Fundamental Limits on Super-Resolution: Light-Fields / Theoretical Properties for Stereo and Face Recognition Textureless Layers

Vision for Safe Driving:

Danger Detection / Prediction and Planning: Robust Car Tracking: Bird's Eye View Generation: Driver Head Tracking: Driver Gaze Estimation

Super-Resolution:

Theoretical Limits: Hallucinating Faces: Super-Resolution Optical Flow

Other:

Projector-Camera Systems / Tele-Graffiti: Catadioptric Camera Design: Feature Detection: The Template Update Problem: Automatic Construction of Active Appearance Models: Setting Low-Level Vision Parameters

I work with a variety of different faculty, students, staff, visitors, and alumni including:

Adrian Broadhurst, Vijayakumar Bhagavatula, German Cheung, Jeff Cohn, Bob Collins, Fernando de la Torre, Ralph Gross, Jessica Hodgins, Changbo Hu, Takahiro Ishikawa, Takeo Kanade, Qifa Ke, Iain Matthews, Andreas Nowatzyk, Raju Patil, Jeff Schneider, Steve Seitz, Jianbo Shi, Terence Sim, Jake Sprouse, Naoya Takao, David Tolliver, Sundar Vedula, Jing Xiao

Please see the projects below for more details.

Research interest keywords

computer vision, object recognition, pattern recognition, sensors, stereo vision, and visual tracking

Past Labs & Groups

Calibrated Imaging Lab - High precision vision and imaging lab
Face Group - Robust detection, recognition, and tracking of human faces with automated analysis of expressions
Human Identification at a Distance - We are developing and evaluating human identification technologies as part of the Defense Advanced Research Projects Agency (DARPA) sponsored program in Human Identification at a Distance (HumanID).
Intelligent Desktop Group - We are developing vision technologies for intelligent desktops.
Virtualized RealityTM - Construct views of real events from nearly any viewpoint
Vision for Safe Driving - Computer vision algorithms and systems for automotive safe driving applications.
Vision for Virtual Environments - Using techniques from computer vision and robotics, we are developing novel sensing and display technologies to support practical, useful virtual environments.
 

Past Projects

2D->3D Face Model Construction - We develop a linear algorithm that uniquely recovers the 3D non-rigid shapes and poses of a human face from a 2D monocular video.
AAM Fitting Algorithms - Many varieties of algorithms for fitting Cootes and Taylor's "Active Appearance Models" are developed.
AAMs with Occlusion - We are developing algorithms to construct AAMs from occluded training images and to efficiently fit AAMs to faces containing occlusion.
Automatic Construction of Active Appearance Models - An algorithm for the automatic (unsupervised) construction of an Active Appearance Model.
Car Tracking - Algorithms for tracking cars and generating "bird's eye views" of the surrounding road scene.
Coplanar Shadowgrams for Acquiring Visual Hulls of Intricate Objects - We present a practical approach to shape-from-silhouettes using a novel technique called coplanar shadowgram imaging that allows us to use dozens to even hundreds of views for visual hull reconstruction.
Face and Facial Feature Tracking - Rigid Tracking of Faces and Non-Rigid Tracking of Facial Features
Face Databases - Miscellaneous face databases collected at CMU.
Face Model Building and Fitting - Techniques for building and fitting 2D and 3D models of human faces and heads.
Face Recognition - Recognizing people from images and videos.
Face Recognition Across Pose - Recognizing people from different poses.
Facial Expression Analysis - Automatic facial expression encoding, extraction and recognition, and expression intensity estimation for the applications of MPEG4 application: teleconferencing, human-computer interaction/interface.
Gaze Estimation - Algorithms for estimating where someone is looking
Hallucinating Faces - A super-resolution algorithm with a strong face-specifc prior.
Human Kinematic Modeling and Motion Capture - We are developing a system for building 3D kinematic models of humans and then using the models to track the person in new video sequences.
Human Motion Transfer - We are developing a system for capturing the motion of one person and rendering a different person performing the same motion.
Image Enhancement for Faces - Video enhancement techniques, specifically tailored for human faces.
Light-fields - A variety of uses of light-fields in computer vision.
Lucas-Kanade 20 Years On - A Unifying Framework for Image Alginment
Photometric Limits on Computer Vision - An investigation into the fundamental limits imposed on computer vision algorithms by imperfect or incomplete photometric information.
PIE Database - A database of 41,368 images of 68 people with Pose, Illumination, and Expression variation.
Prediction & Planning - This project analyses the safety and interaction of moving objects in complex road scenes.
Scene Flow - Methods of computing dense, non-rigid motion of 3D scenes.
Setting Low-Level Vision Parameters - Techniques for feeding back information from high-level vision modules to low-level modules to improve the performance of the overall system.
Spatio-Temporal View Interpolation - An image-based rendering algorithm for view interpolation across both space and time.
Super-Resolution Optical Flow - A super-resolution algorithm for complex non-rigid scenes.
Tele-Graffiti - A system that allows two or more users to communicate remotely via hand-drawn sketches.
Template Update - We are developing an algorithm to update template tracking that avoids the "drifting" problem of the naive update algorithm.
Temporal Shape-From-Silhouette - We are developing algorithms for the computation of 3D shape from multiple silhouette images captured across time.
Textureless Layers - Techniques for the 3D reconstruction of scenes consisting of constant intensity piecewise planar regions (layers).

Selected publications [View all publications]

The Robotics Institute is part of the School of Computer Science, Carnegie Mellon University.
For updates and comments, please see these instructions.
This page maintained by robotwebmaster@ri.cmu.edu