Entropy under Additive Bernoulli and Spherical Noises

Or Ordentlich, Yury Polyanskiy

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

2 Scopus citations


Let Z^{n} be iid Bernoulli (\delta) and U^{n} be uniform on the set of all binary vectors of weight \delta n (Hamming sphere). As is well known, the entropies of Z^{n} and U^{n} are within O(\log n). However, if X^{n} is another binary random variable independent of Z^{n} and U^{n}, we show that H(X^{n}+U^{n}) and H(X^{n}+Z^{n}) are within O(\sqrt{n}) and this estimate is tight. The bound is shown via coupling method. Tightness follows from the observation that the channels x^{n}\mapsto x^{n}+U^{n} and x^{n}\mapsto x^{n}+Z^{n} have similar capacities, but the former has zero dispersion. Finally, we show that despite the \sqrt{n} slack in general, the Mrs. Gerber Lemma for H(X^{n}+U^{n}) holds with only an O(\log n) correction compared to its brethren for H(X^{n}+Z^{n}).

Original languageAmerican English
Title of host publication2018 IEEE International Symposium on Information Theory, ISIT 2018
PublisherInstitute of Electrical and Electronics Engineers Inc.
Number of pages5
ISBN (Print)9781538647806
StatePublished - 15 Aug 2018
Event2018 IEEE International Symposium on Information Theory, ISIT 2018 - Vail, United States
Duration: 17 Jun 201822 Jun 2018

Publication series

NameIEEE International Symposium on Information Theory - Proceedings
ISSN (Print)2157-8095


Conference2018 IEEE International Symposium on Information Theory, ISIT 2018
Country/TerritoryUnited States

Bibliographical note

Publisher Copyright:
© 2018 IEEE.


Dive into the research topics of 'Entropy under Additive Bernoulli and Spherical Noises'. Together they form a unique fingerprint.

Cite this