Synthesizing realistic facial expressions from photographs

Frédéric Pighin; Jamie Hecker; Dani Lischinski; Richard Szeliski; David H. Salesin

doi:10.1145/1185657.1185859

Synthesizing realistic facial expressions from photographs

Frédéric Pighin, Jamie Hecker, Dani Lischinski, Richard Szeliski, David H. Salesin

The Rachel and Selim Benin School of Engineering and Computer Science

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

36 Scopus citations

Abstract

We present new techniques for creating photorealistic textured 3D facial models from photographs of a human subject, and for creating smooth transitions between different facial expressions by morphing between these different models. Starting from several uncalibrated views of a human subject, we employ a user-assisted technique to recover the camera poses corresponding to the views as well as the 3D coordinates of a sparse set of chosen locations on the subject's face. A scattered data interpolation technique is then used to deform a generic face mesh to fit the particular geometry of the subject's face. Having recovered the camera poses and the facial geometry, we extract from the input images one or more texture maps for the model. This process is repeated for several facial expressions of a particular subject. To generate transitions between these facial expressions we use 3D shape morphing between the corresponding face models, while at the same time blending the corresponding textures. Using our technique, we have been able to generate highly realistic face models and natural looking animations.

Original language	English
Title of host publication	SIGGRAPH 2006 - ACM SIGGRAPH 2006 Courses
Publisher	Association for Computing Machinery, Inc
ISBN (Electronic)	1595933646, 9781595933645
DOIs	https://doi.org/10.1145/1185657.1185859
State	Published - 30 Jul 2006
Event	2006 ACM SIGGRAPH Courses, SIGGRAPH 2006 - Boston, United States Duration: 30 Jul 2006 → 3 Aug 2006

Publication series

Name	SIGGRAPH 2006 - ACM SIGGRAPH 2006 Courses

Conference

Conference	2006 ACM SIGGRAPH Courses, SIGGRAPH 2006
Country/Territory	United States
City	Boston
Period	30/07/06 → 3/08/06

Bibliographical note

Funding Information:
We would like to thank Katrin Petersen and Andrew Petty for modeling the generic face model, Cassidy Curtis for his invaluable advice on animating faces, and Joel Auslander and Jason Griffith for early contributions to this project. This work was supported by an NSF Presidential Faculty Fellow award (CCR-9553199), an ONR Young Investigator award (N00014-95-1-0728), and industrial gifts from Microsoft and Pixar.

Keywords

Facial animation
Facial expression generation
Facial modeling
Morphing
Photogrammetry
View-dependent texture-mapping

Access to Document

10.1145/1185657.1185859

Cite this

@inproceedings{02c2c2db2eb04834a861551a530c4f4c,

title = "Synthesizing realistic facial expressions from photographs",

abstract = "We present new techniques for creating photorealistic textured 3D facial models from photographs of a human subject, and for creating smooth transitions between different facial expressions by morphing between these different models. Starting from several uncalibrated views of a human subject, we employ a user-assisted technique to recover the camera poses corresponding to the views as well as the 3D coordinates of a sparse set of chosen locations on the subject's face. A scattered data interpolation technique is then used to deform a generic face mesh to fit the particular geometry of the subject's face. Having recovered the camera poses and the facial geometry, we extract from the input images one or more texture maps for the model. This process is repeated for several facial expressions of a particular subject. To generate transitions between these facial expressions we use 3D shape morphing between the corresponding face models, while at the same time blending the corresponding textures. Using our technique, we have been able to generate highly realistic face models and natural looking animations.",

keywords = "Facial animation, Facial expression generation, Facial modeling, Morphing, Photogrammetry, View-dependent texture-mapping",

author = "Fr{\'e}d{\'e}ric Pighin and Jamie Hecker and Dani Lischinski and Richard Szeliski and Salesin, {David H.}",

note = "Funding Information: We would like to thank Katrin Petersen and Andrew Petty for modeling the generic face model, Cassidy Curtis for his invaluable advice on animating faces, and Joel Auslander and Jason Griffith for early contributions to this project. This work was supported by an NSF Presidential Faculty Fellow award (CCR-9553199), an ONR Young Investigator award (N00014-95-1-0728), and industrial gifts from Microsoft and Pixar.; 2006 ACM SIGGRAPH Courses, SIGGRAPH 2006 ; Conference date: 30-07-2006 Through 03-08-2006",

year = "2006",

month = jul,

day = "30",

doi = "10.1145/1185657.1185859",

language = "אנגלית",

series = "SIGGRAPH 2006 - ACM SIGGRAPH 2006 Courses",

publisher = "Association for Computing Machinery, Inc",

booktitle = "SIGGRAPH 2006 - ACM SIGGRAPH 2006 Courses",

}

Pighin, F, Hecker, J, Lischinski, D, Szeliski, R & Salesin, DH 2006, Synthesizing realistic facial expressions from photographs. in SIGGRAPH 2006 - ACM SIGGRAPH 2006 Courses., 19, SIGGRAPH 2006 - ACM SIGGRAPH 2006 Courses, Association for Computing Machinery, Inc, 2006 ACM SIGGRAPH Courses, SIGGRAPH 2006, Boston, United States, 30/07/06. https://doi.org/10.1145/1185657.1185859

TY - GEN

T1 - Synthesizing realistic facial expressions from photographs

AU - Pighin, Frédéric

AU - Hecker, Jamie

AU - Lischinski, Dani

AU - Szeliski, Richard

AU - Salesin, David H.

N1 - Funding Information: We would like to thank Katrin Petersen and Andrew Petty for modeling the generic face model, Cassidy Curtis for his invaluable advice on animating faces, and Joel Auslander and Jason Griffith for early contributions to this project. This work was supported by an NSF Presidential Faculty Fellow award (CCR-9553199), an ONR Young Investigator award (N00014-95-1-0728), and industrial gifts from Microsoft and Pixar.

PY - 2006/7/30

Y1 - 2006/7/30

N2 - We present new techniques for creating photorealistic textured 3D facial models from photographs of a human subject, and for creating smooth transitions between different facial expressions by morphing between these different models. Starting from several uncalibrated views of a human subject, we employ a user-assisted technique to recover the camera poses corresponding to the views as well as the 3D coordinates of a sparse set of chosen locations on the subject's face. A scattered data interpolation technique is then used to deform a generic face mesh to fit the particular geometry of the subject's face. Having recovered the camera poses and the facial geometry, we extract from the input images one or more texture maps for the model. This process is repeated for several facial expressions of a particular subject. To generate transitions between these facial expressions we use 3D shape morphing between the corresponding face models, while at the same time blending the corresponding textures. Using our technique, we have been able to generate highly realistic face models and natural looking animations.

AB - We present new techniques for creating photorealistic textured 3D facial models from photographs of a human subject, and for creating smooth transitions between different facial expressions by morphing between these different models. Starting from several uncalibrated views of a human subject, we employ a user-assisted technique to recover the camera poses corresponding to the views as well as the 3D coordinates of a sparse set of chosen locations on the subject's face. A scattered data interpolation technique is then used to deform a generic face mesh to fit the particular geometry of the subject's face. Having recovered the camera poses and the facial geometry, we extract from the input images one or more texture maps for the model. This process is repeated for several facial expressions of a particular subject. To generate transitions between these facial expressions we use 3D shape morphing between the corresponding face models, while at the same time blending the corresponding textures. Using our technique, we have been able to generate highly realistic face models and natural looking animations.

KW - Facial animation

KW - Facial expression generation

KW - Facial modeling

KW - Morphing

KW - Photogrammetry

KW - View-dependent texture-mapping

UR - http://www.scopus.com/inward/record.url?scp=85027834891&partnerID=8YFLogxK

U2 - 10.1145/1185657.1185859

DO - 10.1145/1185657.1185859

M3 - ???researchoutput.researchoutputtypes.contributiontobookanthology.conference???

AN - SCOPUS:85027834891

T3 - SIGGRAPH 2006 - ACM SIGGRAPH 2006 Courses

BT - SIGGRAPH 2006 - ACM SIGGRAPH 2006 Courses

PB - Association for Computing Machinery, Inc

T2 - 2006 ACM SIGGRAPH Courses, SIGGRAPH 2006

Y2 - 30 July 2006 through 3 August 2006

ER -

Synthesizing realistic facial expressions from photographs

Abstract

Publication series

Conference

Bibliographical note

Keywords

Access to Document

Other files and links

Fingerprint

Cite this