Abstract
A common assumption in many domains is that high dimensional data are a smooth nonlinear function of a small number of independent factors. When is it possible to recover the factors from unlabeled data? In the context of deep models this problem is called “disentanglement” and was recently shown to be impossible without additional strong assumptions [17, 19]. In this paper, we show that the assumption of local isometry together with non-Gaussianity of the factors, is sufficient to provably recover disentangled representations from data. We leverage recent advances in deep generative models to construct manifolds of highly realistic images for which the ground truth latent representation is known, and test whether modern and classical methods succeed in recovering the latent factors. For many different manifolds, we find that a spectral method that explicitly optimizes local isometry and non-Gaussianity consistently finds the correct latent factors, while baseline deep autoencoders do not. We propose how to encourage deep autoencoders to find encodings that satisfy local isometry and show that this helps them discover disentangled representations. Overall, our results suggest that in some realistic settings, unsupervised disentanglement is provably possible, without any domain-specific assumptions.
Original language | American English |
---|---|
Title of host publication | Advances in Neural Information Processing Systems 34 - 35th Conference on Neural Information Processing Systems, NeurIPS 2021 |
Editors | Marc'Aurelio Ranzato, Alina Beygelzimer, Yann Dauphin, Percy S. Liang, Jenn Wortman Vaughan |
Publisher | Neural information processing systems foundation |
Pages | 5150-5161 |
Number of pages | 12 |
ISBN (Electronic) | 9781713845393 |
State | Published - 2021 |
Event | 35th Conference on Neural Information Processing Systems, NeurIPS 2021 - Virtual, Online Duration: 6 Dec 2021 → 14 Dec 2021 |
Publication series
Name | Advances in Neural Information Processing Systems |
---|---|
Volume | 7 |
ISSN (Print) | 1049-5258 |
Conference
Conference | 35th Conference on Neural Information Processing Systems, NeurIPS 2021 |
---|---|
City | Virtual, Online |
Period | 6/12/21 → 14/12/21 |
Bibliographical note
Funding Information:I would like to thank the New School for Social Research for granting me leave time to do this research; the Fulbright Commission and the Einstein Institution for grants; and the Program on Nonviolent Sanctions and Cultural Survival and the Center for International Affairs, both at Harvard University, for a fellowship. I am also grateful to Ann Dirsa, Lauro Locks, and Colin Naughton for research assistance and to Jorge Domínguez, Jack Hammond, Roger Karapin, William Nylen, Mark Osiel, Pablo Policzer, Sanjay Reddy, Jennifer Schirmer, Charles Tilly, the members of the New School's Proseminar on Political Mobilization and Conflict, and six anonymous LARR reviewers for comments on earlier drafts of this article.
Publisher Copyright:
© 2021 Neural information processing systems foundation. All rights reserved.