When is Unsupervised Disentanglement Possible?

Daniella Horan, Eitan Richardson, Yair Weiss

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

8 Scopus citations


A common assumption in many domains is that high dimensional data are a smooth nonlinear function of a small number of independent factors. When is it possible to recover the factors from unlabeled data? In the context of deep models this problem is called “disentanglement” and was recently shown to be impossible without additional strong assumptions [17, 19]. In this paper, we show that the assumption of local isometry together with non-Gaussianity of the factors, is sufficient to provably recover disentangled representations from data. We leverage recent advances in deep generative models to construct manifolds of highly realistic images for which the ground truth latent representation is known, and test whether modern and classical methods succeed in recovering the latent factors. For many different manifolds, we find that a spectral method that explicitly optimizes local isometry and non-Gaussianity consistently finds the correct latent factors, while baseline deep autoencoders do not. We propose how to encourage deep autoencoders to find encodings that satisfy local isometry and show that this helps them discover disentangled representations. Overall, our results suggest that in some realistic settings, unsupervised disentanglement is provably possible, without any domain-specific assumptions.

Original languageAmerican English
Title of host publicationAdvances in Neural Information Processing Systems 34 - 35th Conference on Neural Information Processing Systems, NeurIPS 2021
EditorsMarc'Aurelio Ranzato, Alina Beygelzimer, Yann Dauphin, Percy S. Liang, Jenn Wortman Vaughan
PublisherNeural information processing systems foundation
Number of pages12
ISBN (Electronic)9781713845393
StatePublished - 2021
Event35th Conference on Neural Information Processing Systems, NeurIPS 2021 - Virtual, Online
Duration: 6 Dec 202114 Dec 2021

Publication series

NameAdvances in Neural Information Processing Systems
ISSN (Print)1049-5258


Conference35th Conference on Neural Information Processing Systems, NeurIPS 2021
CityVirtual, Online

Bibliographical note

Funding Information:
I would like to thank the New School for Social Research for granting me leave time to do this research; the Fulbright Commission and the Einstein Institution for grants; and the Program on Nonviolent Sanctions and Cultural Survival and the Center for International Affairs, both at Harvard University, for a fellowship. I am also grateful to Ann Dirsa, Lauro Locks, and Colin Naughton for research assistance and to Jorge Domínguez, Jack Hammond, Roger Karapin, William Nylen, Mark Osiel, Pablo Policzer, Sanjay Reddy, Jennifer Schirmer, Charles Tilly, the members of the New School's Proseminar on Political Mobilization and Conflict, and six anonymous LARR reviewers for comments on earlier drafts of this article.

Publisher Copyright:
© 2021 Neural information processing systems foundation. All rights reserved.

Cite this