Asymptotics of the entropy rate for a Hidden Markov Process

Or Zuk*, Ido Kanter, Eytan Domany

*Corresponding author for this work

Research output: Contribution to journalConference articlepeer-review

6 Scopus citations


We calculate the Shannon entropy rate of a binary Hidden Markov Process (HMP), of given transition rate and noise ε (emission), as a series expansion in ε. The first two orders are calculated exactly. We then evaluate, for finite histories, simple upper-bounds of Cover and Thomas. Surprisingly, we find that for a fixed order k and history of n steps, the bounds become independent of n for large enough n. This observation is the basis of a conjecture, that the upper-bound obtained for n ≥ (k + 3)/2 gives the exact entropy rate for any desired order k of ε.

Original languageAmerican English
Pages (from-to)173-182
Number of pages10
JournalProceedings of the Data Compression Conference
StatePublished - 2005
Externally publishedYes
EventDCC 2005: Data Compression Conference - Snowbird, UT, United States
Duration: 29 Mar 200531 Mar 2005


Dive into the research topics of 'Asymptotics of the entropy rate for a Hidden Markov Process'. Together they form a unique fingerprint.

Cite this