/2D->3D Face Model Construction

2D->3D Face Model Construction

Portrait of 2D->3D Face Model Construction
Head: Jing Xiao
Contact: Jing Xiao
Last Project Publication Year: 2004

We develop a linear algorithm that uniquely recovers the 3D non-rigid shapes and poses of a human face from a 2D monocular video. The non-rigid shapes can be regarded as a weighted combination of certain shape bases. We thus construct a 3D morphable model of the face using the recovered shape bases. Previous work on this problem utilized only orthonormality constraints on the camera rotations (rotation constraints). We prove that using only the rotation constraints results in ambiguous and invalid solutions. To eliminate the ambiguity, we present a set of novel constraints on the shape bases (basis constraints). It is proved that, under the weak-perspective projection model and the non-degenerate situations, enforcing both the basis and rotation constraints leads to a closed-form solution to reconstructing the non-rigid shape and motion. The accuracy and robustness of the approach is evaluated quantitatively on synthetic data and qualitatively on real video sequences. Once we have constructed a 3D face model, it can be used to track the face in 3D in real-time using our 3D AAM fitting algorithms.

The upper-left window shows the input video with tracked feature points. It contains head rotation and a few facial expressions.
The upper-middle window demonstrates the reconstructed 3D meshes formed by the feature points.
The upper-right window shows the reconstructed 3D locations of those feature points.

According to the reconstruction, we stabilized the face appearance into both the profile views and the frontal view, as shown in the lower windows. The non-rigid facial deformations such as mouth opening and eye closure were recovered.

Displaying 3 Publications
Non-Rigid Shape and Motion Recovery: Degenerate Deformations
Jing Xiao and Takeo Kanade

Conference Paper, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Vol. 1, pp. 668 - 675, June, 2004
Real-Time Combined 2D+3D Active Appearance Models
Jing Xiao, Simon Baker, Iain Matthews and Takeo Kanade

Conference Paper, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Vol. 2, pp. 535 - 542, June, 2004
A Closed-Form Solution to Non-Rigid Shape and Motion Recovery
Jing Xiao, Jinxiang Chai and Takeo Kanade

Conference Paper, The 8th European Conference on Computer Vision (ECCV 2004), May, 2004

Current RI People

Past Project People