A connected auto-encoders based approach for image separation with side information: With applications to art investigation

Wei Pu, Barak Sober, Nathan Daly, Catherine Higgitt, Ingrid Daubechies, Miguel R.D. Rodrigues

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

6 Scopus citations

Abstract

X-radiography is a widely used imaging technique in art investigation, whether to investigate the condition of a painting or provide insights into artists' techniques and working methods. In this paper, we propose a new architecture based on the use of 'connected' auto-encoders in order to separate mixed X-ray images acquired from double-sided paintings, where in addition to the mixed X-ray image one can also exploit the two RGB images associated with the front and back of the painting. This proposed architecture uses convolutional autoencoders that extract features from the RGB images that can be employed to (1) reproduce both of the original RGB images, (2) reconstruct the associated separated X-ray images, and (3) regenerate the mixed X-ray image. It operates in a totally self-supervised fashion without the need for examples containing both the mixed X-ray images and the separated ones. Based on images from the double-sided wing panels from the famous Ghent Altarpiece, painted in 1432 by the brothers Hubert and Jan Van Eyck, the proposed algorithm has been experimentally verified to outperform state-of-theart X-ray separation methods in art investigation applications.

Original languageAmerican English
Title of host publication2020 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2020 - Proceedings
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages2213-2217
Number of pages5
ISBN (Electronic)9781509066315
DOIs
StatePublished - May 2020
Externally publishedYes
Event2020 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2020 - Barcelona, Spain
Duration: 4 May 20208 May 2020

Publication series

NameICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings
Volume2020-May
ISSN (Print)1520-6149

Conference

Conference2020 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2020
Country/TerritorySpain
CityBarcelona
Period4/05/208/05/20

Bibliographical note

Publisher Copyright:
© 2020 Institute of Electrical and Electronics Engineers Inc.. All rights reserved.

Keywords

  • Autoencoders
  • Convolutional neural networks
  • Deep neural networks
  • Image separation with side information

Fingerprint

Dive into the research topics of 'A connected auto-encoders based approach for image separation with side information: With applications to art investigation'. Together they form a unique fingerprint.

Cite this