Critical Slowing down near Topological Transitions in Rate-Distortion Problems

Shlomi Agrnon, Etam Benger, Or Ordentlich, Naftali Tishby

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

3 Scopus citations

Abstract

In rate-distortion (RD) problems one seeks reduced representations of a source that meet a target distortion constraint. Such optimal representations undergo topological transitions at some critical rate values, when their cardinality or dimensionality change. We study the convergence time of the Arimoto-Blahut alternating projection algorithms, used to solve such problems, near those critical points, both for the ratedistortion and information bottleneck settings. We argue that they suffer from critical slowing down - a diverging number of iterations for convergence - near the critical points. This phenomenon can have theoretical and practical implications for both machine learning and data compression problems.

Original languageAmerican English
Title of host publication2021 IEEE International Symposium on Information Theory, ISIT 2021 - Proceedings
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages2625-2630
Number of pages6
ISBN (Electronic)9781538682098
DOIs
StatePublished - 12 Jul 2021
Event2021 IEEE International Symposium on Information Theory, ISIT 2021 - Virtual, Melbourne, Australia
Duration: 12 Jul 202120 Jul 2021

Publication series

NameIEEE International Symposium on Information Theory - Proceedings
Volume2021-July
ISSN (Print)2157-8095

Conference

Conference2021 IEEE International Symposium on Information Theory, ISIT 2021
Country/TerritoryAustralia
CityVirtual, Melbourne
Period12/07/2120/07/21

Bibliographical note

Publisher Copyright:
© 2021 IEEE.

Fingerprint

Dive into the research topics of 'Critical Slowing down near Topological Transitions in Rate-Distortion Problems'. Together they form a unique fingerprint.

Cite this