The Double-Sided Information-Bottleneck Function

Michael Dikshtein, Or Ordentlich, Shlomo Shamai Shitz

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

We consider a two-terminal variant (double-sided) of the information bottleneck problem, which is related to biclus-tering. In our setup, x and Y are dependent random variables and the problem is to find two independent channels \mathrm{P}_{\cup 1\times} and \mathrm{p}_{\vee 1!} (setting the Markovian structure \cup\rightarrow\times\rightarrow \mathrm{Y}\rightarrow V) that maximize I(\cup;\mathrm{V}) subject to constraints on the relevant mutual information expressions: I(\cup;\mathrm{X}) and I(\mathrm{V};\mathrm{Y}). For jointly Gaussian X and Y, we show that Gaussian channels are optimal in the low-SNR regime, but not for general SNR. Similarly, it is shown that for a doubly symmetric binary source, binary symmetric channels are optimal when the correlation is low, and are suboptimal for high correlation. We conjecture that Z and S channels are optimal when the correlation is 1 (i.e., \mathrm{X}=\mathrm{Y}), and provide supporting numerical evidence.

Original languageAmerican English
Title of host publication2021 IEEE International Symposium on Information Theory, ISIT 2021 - Proceedings
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages2495-2500
Number of pages6
ISBN (Electronic)9781538682098
DOIs
StatePublished - 12 Jul 2021
Event2021 IEEE International Symposium on Information Theory, ISIT 2021 - Virtual, Melbourne, Australia
Duration: 12 Jul 202120 Jul 2021

Publication series

NameIEEE International Symposium on Information Theory - Proceedings
Volume2021-July
ISSN (Print)2157-8095

Conference

Conference2021 IEEE International Symposium on Information Theory, ISIT 2021
Country/TerritoryAustralia
CityVirtual, Melbourne
Period12/07/2120/07/21

Bibliographical note

Publisher Copyright:
© 2021 IEEE.

Fingerprint

Dive into the research topics of 'The Double-Sided Information-Bottleneck Function'. Together they form a unique fingerprint.

Cite this