Abstract
In rate-distortion (RD) problems one seeks reduced representations of a source that meet a target distortion constraint. Such optimal representations undergo topological transitions at some critical rate values, when their cardinality or dimensionality change. We study the convergence time of the Arimoto-Blahut alternating projection algorithms, used to solve such problems, near those critical points, both for the ratedistortion and information bottleneck settings. We argue that they suffer from critical slowing down - a diverging number of iterations for convergence - near the critical points. This phenomenon can have theoretical and practical implications for both machine learning and data compression problems.
| Original language | American English |
|---|---|
| Title of host publication | 2021 IEEE International Symposium on Information Theory, ISIT 2021 - Proceedings |
| Publisher | Institute of Electrical and Electronics Engineers Inc. |
| Pages | 2625-2630 |
| Number of pages | 6 |
| ISBN (Electronic) | 9781538682098 |
| DOIs | |
| State | Published - 12 Jul 2021 |
| Event | 2021 IEEE International Symposium on Information Theory, ISIT 2021 - Virtual, Melbourne, Australia Duration: 12 Jul 2021 → 20 Jul 2021 |
Publication series
| Name | IEEE International Symposium on Information Theory - Proceedings |
|---|---|
| Volume | 2021-July |
| ISSN (Print) | 2157-8095 |
Conference
| Conference | 2021 IEEE International Symposium on Information Theory, ISIT 2021 |
|---|---|
| Country/Territory | Australia |
| City | Virtual, Melbourne |
| Period | 12/07/21 → 20/07/21 |
Bibliographical note
Funding Information:This work was supported in part by the Gatsby Foundation and in part by the Pazy Foundation. E.B. is further supported by the Center for Interdisciplinary Data Science Research (CIDR) at the Hebrew University. The work of O.O. was supported by the ISF under Grant 1791/17.
Publisher Copyright:
© 2021 IEEE.