Fine Asymptotics for Universal One-to-One Compression of Parametric Sources

Nematollah Iri, Oliver Kosut

Research output: Contribution to journalArticlepeer-review

5 Scopus citations


Universal source coding at short blocklengths is considered for an i.i.d. exponential family of distributions. The Type Size code has previously been shown to be optimal up to the third-order rate for universal compression of all memoryless sources over finite alphabets. The Type Size code assigns sequences ordered based on their type class sizes to binary strings ordered lexicographically. To generalize this type class approach for parametric sources, a natural scheme is to define two sequences to be in the same type class if and only if they are equiprobable under any model in the parametric class. This natural approach, however, is shown to be suboptimal. A variation of the Type Size code is introduced, where type classes are defined based on neighborhoods of minimal sufficient statistics. The asymptotics of the overflow rate of this variation are derived and a converse result establishes its optimality up to the third-order term.

Original languageEnglish (US)
Article number8638791
Pages (from-to)2442-2458
Number of pages17
JournalIEEE Transactions on Information Theory
Issue number4
StatePublished - Apr 2019


  • Data compression
  • finite blocklength
  • parametric statistics
  • universal source coding

ASJC Scopus subject areas

  • Information Systems
  • Computer Science Applications
  • Library and Information Sciences


Dive into the research topics of 'Fine Asymptotics for Universal One-to-One Compression of Parametric Sources'. Together they form a unique fingerprint.

Cite this