Entropy under Additive Bernoulli and Spherical Noises

Or Ordentlich, Yury Polyanskiy

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

2 Scopus citations

Abstract

Let Z^{n} be iid Bernoulli (\delta) and U^{n} be uniform on the set of all binary vectors of weight \delta n (Hamming sphere). As is well known, the entropies of Z^{n} and U^{n} are within O(\log n). However, if X^{n} is another binary random variable independent of Z^{n} and U^{n}, we show that H(X^{n}+U^{n}) and H(X^{n}+Z^{n}) are within O(\sqrt{n}) and this estimate is tight. The bound is shown via coupling method. Tightness follows from the observation that the channels x^{n}\mapsto x^{n}+U^{n} and x^{n}\mapsto x^{n}+Z^{n} have similar capacities, but the former has zero dispersion. Finally, we show that despite the \sqrt{n} slack in general, the Mrs. Gerber Lemma for H(X^{n}+U^{n}) holds with only an O(\log n) correction compared to its brethren for H(X^{n}+Z^{n}).

Original languageEnglish
Title of host publication2018 IEEE International Symposium on Information Theory, ISIT 2018
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages521-525
Number of pages5
ISBN (Print)9781538647806
DOIs
StatePublished - 15 Aug 2018
Event2018 IEEE International Symposium on Information Theory, ISIT 2018 - Vail, United States
Duration: 17 Jun 201822 Jun 2018

Publication series

NameIEEE International Symposium on Information Theory - Proceedings
Volume2018-June
ISSN (Print)2157-8095

Conference

Conference2018 IEEE International Symposium on Information Theory, ISIT 2018
Country/TerritoryUnited States
CityVail
Period17/06/1822/06/18

Bibliographical note

Publisher Copyright:
© 2018 IEEE.

Fingerprint

Dive into the research topics of 'Entropy under Additive Bernoulli and Spherical Noises'. Together they form a unique fingerprint.

Cite this