Synthesizing training images for boosting human 3D pose estimation

Wenzheng Chen*, Huan Wang, Yangyan Li, Hao Su, Zhenhua Wang, Changhe Tu, Dani Lischinski, Daniel Cohen-Or, Baoquan Chen

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

193 Scopus citations


Human 3D pose estimation from a single image is a challenging task with numerous applications. Convolutional Neural Networks (CNNs) have recently achieved superior performance on the task of 2D pose estimation from a single image, by training on images with 2D annotations collected by crowd sourcing. This suggests that similar success could be achieved for direct estimation of 3D poses. However, 3D poses are much harder to annotate, and the lack of suitable annotated training images hinders attempts towards end-to-end solutions. To address this issue, we opt to automatically synthesize training images with ground truth pose annotations. Our work is a systematic study along this road. We find that pose space coverage and texture diversity are the key ingredients for the effectiveness of synthetic training data. We present a fully automatic, scalable approach that samples the human pose space for guiding the synthesis procedure and extracts clothing textures from real images. Furthermore, we explore domain adaptation for bridging the gap between our synthetic training images and real testing photos. We demonstrate that CNNs trained with our synthetic images out-perform those trained with real photos on 3D pose estimation tasks.

Original languageAmerican English
Title of host publicationProceedings - 2016 4th International Conference on 3D Vision, 3DV 2016
PublisherInstitute of Electrical and Electronics Engineers Inc.
Number of pages10
ISBN (Electronic)9781509054077
StatePublished - 15 Dec 2016
Event4th International Conference on 3D Vision, 3DV 2016 - Stanford, United States
Duration: 25 Oct 201628 Oct 2016

Publication series

NameProceedings - 2016 4th International Conference on 3D Vision, 3DV 2016


Conference4th International Conference on 3D Vision, 3DV 2016
Country/TerritoryUnited States

Bibliographical note

Publisher Copyright:
© 2016 IEEE.


  • Deep learning
  • Domain adaptation
  • Human 3D pose
  • Synthesizing training data


Dive into the research topics of 'Synthesizing training images for boosting human 3D pose estimation'. Together they form a unique fingerprint.

Cite this