The Double-Sided Information Bottleneck Function †

Michael Dikshtein*, Or Ordentlich, Shlomo Shamai

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

Abstract

A double-sided variant of the information bottleneck method is considered. Let (X,Y) be a bivariate source characterized by a joint pmf PXY. The problem is to find two independent channels PU|X and PV|Y (setting the Markovian structure U→X→Y→V), that maximize I(U;V) subject to constraints on the relevant mutual information expressions: I(U;X) and I(V;Y). For jointly Gaussian X and Y, we show that Gaussian channels are optimal in the low-SNR regime but not for general SNR. Similarly, it is shown that for a doubly symmetric binary source, binary symmetric channels are optimal when the correlation is low and are suboptimal for high correlations. We conjecture that Z and S channels are optimal when the correlation is 1 (i.e., X=Y) and provide supporting numerical evidence. Furthermore, we present a Blahut-Arimoto type alternating maximization algorithm and demonstrate its performance for a representative setting. This problem is closely related to the domain of biclustering.

Original languageAmerican English
Article number1321
JournalEntropy
Volume24
Issue number9
DOIs
StatePublished - 19 Sep 2022

Bibliographical note

Publisher Copyright:
© 2022 by the authors.

Keywords

  • biclustering
  • information bottleneck
  • lossy compression
  • remote source coding

Fingerprint

Dive into the research topics of 'The Double-Sided Information Bottleneck Function †'. Together they form a unique fingerprint.

Cite this