A connected auto-encoders based approach for image separation with side information: With applications to art investigation

X-radiography is a widely used imaging technique in art investigation, whether to investigate the condition of a painting or provide insights into artists' techniques and working methods. In this paper, we propose a new architecture based on the use of 'connected' auto-encoders in order to separate mixed X-ray images acquired from double-sided paintings, where in addition to the mixed X-ray image one can also exploit the two RGB images associated with the front and back of the painting. This proposed architecture uses convolutional autoencoders that extract features from the RGB images that can be employed to (1) reproduce both of the original RGB images, (2) reconstruct the associated separated X-ray images, and (3) regenerate the mixed X-ray image. It operates in a totally self-supervised fashion without the need for examples containing both the mixed X-ray images and the separated ones. Based on images from the double-sided wing panels from the famous Ghent Altarpiece, painted in 1432 by the brothers Hubert and Jan Van Eyck, the proposed algorithm has been experimentally verified to outperform state-of-theart X-ray separation methods in art investigation applications.