Novel lower bounds on the entropy rate of binary hidden Markov processes

Or Ordentlich

doi:10.1109/ISIT.2016.7541387

Novel lower bounds on the entropy rate of binary hidden Markov processes

Or Ordentlich^*

^*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

10 Scopus citations

Abstract

Recently, Samorodnitsky proved a strengthened version of Mrs. Gerber's Lemma, where the output entropy of a binary symmetric channel is bounded in terms of the average entropy of the input projected on a random subset of coordinates. Here, this result is applied for deriving novel lower bounds on the entropy rate of binary hidden Markov processes. For symmetric underlying Markov processes, our bound improves upon the best known bound in the very noisy regime. The nonsymmetric case is also considered, and explicit bounds are derived for Markov processes that satisfy the (1, ∞)-RLL constraint.

Original language	English
Title of host publication	Proceedings - ISIT 2016; 2016 IEEE International Symposium on Information Theory
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	690-694
Number of pages	5
ISBN (Electronic)	9781509018062
DOIs	https://doi.org/10.1109/ISIT.2016.7541387
State	Published - 10 Aug 2016
Externally published	Yes
Event	2016 IEEE International Symposium on Information Theory, ISIT 2016 - Barcelona, Spain Duration: 10 Jul 2016 → 15 Jul 2016

Publication series

Name	IEEE International Symposium on Information Theory - Proceedings
Volume	2016-August
ISSN (Print)	2157-8095

Conference

Conference	2016 IEEE International Symposium on Information Theory, ISIT 2016
Country/Territory	Spain
City	Barcelona
Period	10/07/16 → 15/07/16

Bibliographical note

Publisher Copyright:
© 2016 IEEE.

Access to Document

10.1109/ISIT.2016.7541387

Cite this

Ordentlich, O. (2016). Novel lower bounds on the entropy rate of binary hidden Markov processes. In Proceedings - ISIT 2016; 2016 IEEE International Symposium on Information Theory (pp. 690-694). Article 7541387 (IEEE International Symposium on Information Theory - Proceedings; Vol. 2016-August). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ISIT.2016.7541387

@inproceedings{2b767391e4a24244bcf0c83fdf82073a,

title = "Novel lower bounds on the entropy rate of binary hidden Markov processes",

abstract = "Recently, Samorodnitsky proved a strengthened version of Mrs. Gerber's Lemma, where the output entropy of a binary symmetric channel is bounded in terms of the average entropy of the input projected on a random subset of coordinates. Here, this result is applied for deriving novel lower bounds on the entropy rate of binary hidden Markov processes. For symmetric underlying Markov processes, our bound improves upon the best known bound in the very noisy regime. The nonsymmetric case is also considered, and explicit bounds are derived for Markov processes that satisfy the (1, ∞)-RLL constraint.",

author = "Or Ordentlich",

note = "Publisher Copyright: {\textcopyright} 2016 IEEE.; 2016 IEEE International Symposium on Information Theory, ISIT 2016 ; Conference date: 10-07-2016 Through 15-07-2016",

year = "2016",

month = aug,

day = "10",

doi = "10.1109/ISIT.2016.7541387",

language = "אנגלית",

series = "IEEE International Symposium on Information Theory - Proceedings",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "690--694",

booktitle = "Proceedings - ISIT 2016; 2016 IEEE International Symposium on Information Theory",

address = "ארצות הברית",

}

Ordentlich, O 2016, Novel lower bounds on the entropy rate of binary hidden Markov processes. in Proceedings - ISIT 2016; 2016 IEEE International Symposium on Information Theory., 7541387, IEEE International Symposium on Information Theory - Proceedings, vol. 2016-August, Institute of Electrical and Electronics Engineers Inc., pp. 690-694, 2016 IEEE International Symposium on Information Theory, ISIT 2016, Barcelona, Spain, 10/07/16. https://doi.org/10.1109/ISIT.2016.7541387

Novel lower bounds on the entropy rate of binary hidden Markov processes. / Ordentlich, Or.
Proceedings - ISIT 2016; 2016 IEEE International Symposium on Information Theory. Institute of Electrical and Electronics Engineers Inc., 2016. p. 690-694 7541387 (IEEE International Symposium on Information Theory - Proceedings; Vol. 2016-August).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Novel lower bounds on the entropy rate of binary hidden Markov processes

AU - Ordentlich, Or

PY - 2016/8/10

Y1 - 2016/8/10

N2 - Recently, Samorodnitsky proved a strengthened version of Mrs. Gerber's Lemma, where the output entropy of a binary symmetric channel is bounded in terms of the average entropy of the input projected on a random subset of coordinates. Here, this result is applied for deriving novel lower bounds on the entropy rate of binary hidden Markov processes. For symmetric underlying Markov processes, our bound improves upon the best known bound in the very noisy regime. The nonsymmetric case is also considered, and explicit bounds are derived for Markov processes that satisfy the (1, ∞)-RLL constraint.

AB - Recently, Samorodnitsky proved a strengthened version of Mrs. Gerber's Lemma, where the output entropy of a binary symmetric channel is bounded in terms of the average entropy of the input projected on a random subset of coordinates. Here, this result is applied for deriving novel lower bounds on the entropy rate of binary hidden Markov processes. For symmetric underlying Markov processes, our bound improves upon the best known bound in the very noisy regime. The nonsymmetric case is also considered, and explicit bounds are derived for Markov processes that satisfy the (1, ∞)-RLL constraint.

UR - http://www.scopus.com/inward/record.url?scp=84985930528&partnerID=8YFLogxK

U2 - 10.1109/ISIT.2016.7541387

DO - 10.1109/ISIT.2016.7541387

M3 - ???researchoutput.researchoutputtypes.contributiontobookanthology.conference???

AN - SCOPUS:84985930528

T3 - IEEE International Symposium on Information Theory - Proceedings

SP - 690

EP - 694

BT - Proceedings - ISIT 2016; 2016 IEEE International Symposium on Information Theory

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 2016 IEEE International Symposium on Information Theory, ISIT 2016

Y2 - 10 July 2016 through 15 July 2016

ER -

Novel lower bounds on the entropy rate of binary hidden Markov processes

Abstract

Publication series

Conference

Bibliographical note

Access to Document

Other files and links

Fingerprint

Cite this