Appearance modeling via proxy-to-image alignment

Hui Huang*, Ke Xie, Lin Ma, Dani Lischinski, Minglun Gong, Xin Tong, Daniel Cohen-Or

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

7 Scopus citations

Abstract

Endowing 3D objects with realistic surface appearance is a challenging and time-demanding task, as real-world surfaces typically exhibit a plethora of spatially variant geometric and photometric detail. Not surprisingly, computer artists commonly use images of real-world objects as an inspiration and a reference for their digital creations. However, despite two decades of research on image-based modeling, there are still no tools available for automatically extracting the detailed appearance (microgeometry and texture) of a 3D surface from a single image. In this article, we present a novel user-assisted approach for quickly and easily extracting a nonparametric appearance model from a single photograph of a reference object. The extraction process requires a user-provided proxy, whose geometry roughly approximates that of the object in the image. Since the proxy is just a rough approximation, it is necessary to align and deform it so as to match the reference object. The main contribution of this work is a novel technique to perform such an alignment, which enables accurate joint recovery of geometric detail and reflectance. The correlations between the recovered geometry at various scales and the spatially varying reflectance constitute a nonparametric appearance model. Once extracted, the appearance model may then be applied to various 3D shapes, whose large-scale geometry may differ considerably from that of the original reference object. Thus, our approach makes it possible to construct an appearance library, allowing users to easily enrich detail-less 3D shapes with realistic geometric detail and surface texture.

Original languageAmerican English
Article number10
JournalACM Transactions on Graphics
Volume37
Issue number1
DOIs
StatePublished - Jan 2018

Bibliographical note

Publisher Copyright:
© 2017 ACM.

Keywords

  • Appearance modeling
  • Appearance transfer
  • Detail extraction
  • Intrinsic image decomposition
  • Shape alignment
  • Shape deformation
  • Texture synthesis

Fingerprint

Dive into the research topics of 'Appearance modeling via proxy-to-image alignment'. Together they form a unique fingerprint.

Cite this