Differentiable refraction-tracing for mesh reconstruction of transparent objects

Jiahui Lyu, Bojian Wu, Dani Lischinski, Daniel Cohen-Or, Hui Huang*

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

17 Scopus citations


Capturing the 3D geometry of transparent objects is a challenging task, ill-suited for general-purpose scanning and reconstruction techniques, since these cannot handle specular light transport phenomena. Existing state-of-the-art methods, designed specifically for this task, either involve a complex setup to reconstruct complete refractive ray paths, or leverage a data-driven approach based on synthetic training data. In either case, the reconstructed 3D models suffer from over-smoothing and loss of fine detail. This paper introduces a novel, high precision, 3D acquisition and reconstruction method for solid transparent objects. Using a static background with a coded pattern, we establish a mapping between the camera view rays and locations on the background. Differentiable tracing of refractive ray paths is then used to directly optimize a 3D mesh approximation of the object, while simultaneously ensuring silhouette consistency and smoothness. Extensive experiments and comparisons demonstrate the superior accuracy of our method.

Original languageAmerican English
Article number195
JournalACM Transactions on Graphics
Issue number6
StatePublished - 26 Nov 2020

Bibliographical note

Publisher Copyright:
© 2020 ACM.


  • 3D reconstruction
  • differentiable rendering
  • transparent objects


Dive into the research topics of 'Differentiable refraction-tracing for mesh reconstruction of transparent objects'. Together they form a unique fingerprint.

Cite this