Limit results on pattern entropy

Alon Orlitsky, Narayana P. Santhanam, Krishnamurthy Viswanathan, Junan Zhang

Research output: Contribution to journalArticlepeer-review

18 Scopus citations


We determine the entropy rate of patterns of certain random processes including all finite-entropy stationary processes. For independent and identically distributed (i.i.d.) processes, we also bound the speed at which the per-symbol pattern entropy converges to this rate, and show that patterns satisfy an asymptotic equipartition property. To derive some of these results we upper bound the probability that the nth variable in a random process differs from all preceding ones. copy; 2006 IEEE.

Original languageEnglish (US)
Pages (from-to)2954-2964
Number of pages11
JournalIEEE Transactions on Information Theory
Issue number7
StatePublished - Jul 2006
Externally publishedYes


  • Asymptotic equipartition property
  • Entropy rate
  • Large alphabets
  • Patterns
  • Universal compression

ASJC Scopus subject areas

  • Information Systems
  • Computer Science Applications
  • Library and Information Sciences


Dive into the research topics of 'Limit results on pattern entropy'. Together they form a unique fingerprint.

Cite this