Generating 3D Human Animations from Single Monocular Images

Tanya Marwah
Master's Thesis, Tech. Report, CMU-RI-TR-19-12, May, 2019

Download Publication

Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.


Endowing AI systems with the ability to formulate a three-dimensional understanding of human appearance from a single RGB image is an important component technology for applications such as person re-identification, biometrics, virtual reality and augmented reality. However, jointly inferring the texture map and 3D human model from a single image is challenging because: (1) scale ambiguity, clothing and occlusion make it hard to obtain the 3D model of a person from a single monocular image; and (2) a complete texture map needs to be inferred for each part of the body, such as skin tone, clothing and facial details, even if parts are occluded or are not visible in the image. With these challenges in mind, we introduce a methodology to generate the complete texture of a 3D model of a human from a single image. In particular, we use a model-based approach for estimating the body-shape and pose of the 3D mesh model and concurrently utilize it towards a data-driven approach for estimating the UV texture map of the body (e.g., shirt, pants, skin tone, face).

We also show how our parameterization of texture and human pose are widely applicable and can easily be extended to generate animations based on learned humanoid control policies and hence can be utilized for video generation. Furthermore, the textures generated by our method are compatible with existing graphics engines such as Blender, Unity and Unreal Engine and hence can be utilized by animators and content creators for a wide range of applications

author = {Tanya Marwah},
title = {Generating 3D Human Animations from Single Monocular Images},
year = {2019},
month = {May},
school = {},
address = {Pittsburgh, PA},
number = {CMU-RI-TR-19-12},
} 2019-05-13T12:34:02-04:00