Estimation information bounds using the I-MMSE formula and Gaussian mixture models

Bryan Paul; Daniel Bliss

doi:10.1109/CISS.2016.7460514

Estimation information bounds using the I-MMSE formula and Gaussian mixture models

Bryan Paul, Daniel Bliss

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

8 Scopus citations

Abstract

We derive a method to bound the mutual information between a noisy and noiseless measurement exploiting the I-MMSE estimation and information theory connection. Modeling the source distribution as a Gaussian mixture model, a closed form expression for upper and lower bounds of the minimum mean square error is found using recent results. Using the connection between rate of information relative to SNR and the minimum mean square error of the estimator, the mutual information can be bounded as well for arbitrary source distributions in Gaussian noise.

Original language	English (US)
Title of host publication	2016 50th Annual Conference on Information Systems and Sciences, CISS 2016
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	274-279
Number of pages	6
ISBN (Electronic)	9781467394574
DOIs	https://doi.org/10.1109/CISS.2016.7460514
State	Published - Apr 26 2016
Event	50th Annual Conference on Information Systems and Sciences, CISS 2016 - Princeton, United States Duration: Mar 16 2016 → Mar 18 2016

Other

Other	50th Annual Conference on Information Systems and Sciences, CISS 2016
Country/Territory	United States
City	Princeton
Period	3/16/16 → 3/18/16

Keywords

Bounds
Estimation Information
Gaussian Mixture Models
I-MMSE

ASJC Scopus subject areas

Computer Networks and Communications
Information Systems

Access to Document

10.1109/CISS.2016.7460514

Cite this

Paul, B & Bliss, D 2016, Estimation information bounds using the I-MMSE formula and Gaussian mixture models. in 2016 50th Annual Conference on Information Systems and Sciences, CISS 2016., 7460514, Institute of Electrical and Electronics Engineers Inc., pp. 274-279, 50th Annual Conference on Information Systems and Sciences, CISS 2016, Princeton, United States, 3/16/16. https://doi.org/10.1109/CISS.2016.7460514

@inproceedings{b32d03343e8245378b8b74f7ea3e1f8d,

title = "Estimation information bounds using the I-MMSE formula and Gaussian mixture models",

abstract = "We derive a method to bound the mutual information between a noisy and noiseless measurement exploiting the I-MMSE estimation and information theory connection. Modeling the source distribution as a Gaussian mixture model, a closed form expression for upper and lower bounds of the minimum mean square error is found using recent results. Using the connection between rate of information relative to SNR and the minimum mean square error of the estimator, the mutual information can be bounded as well for arbitrary source distributions in Gaussian noise.",

keywords = "Bounds, Estimation Information, Gaussian Mixture Models, I-MMSE",

author = "Bryan Paul and Daniel Bliss",

year = "2016",

month = apr,

day = "26",

doi = "10.1109/CISS.2016.7460514",

language = "English (US)",

pages = "274--279",

booktitle = "2016 50th Annual Conference on Information Systems and Sciences, CISS 2016",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

address = "United States",

note = "50th Annual Conference on Information Systems and Sciences, CISS 2016 ; Conference date: 16-03-2016 Through 18-03-2016",

}

TY - GEN

T1 - Estimation information bounds using the I-MMSE formula and Gaussian mixture models

AU - Paul, Bryan

AU - Bliss, Daniel

PY - 2016/4/26

Y1 - 2016/4/26

N2 - We derive a method to bound the mutual information between a noisy and noiseless measurement exploiting the I-MMSE estimation and information theory connection. Modeling the source distribution as a Gaussian mixture model, a closed form expression for upper and lower bounds of the minimum mean square error is found using recent results. Using the connection between rate of information relative to SNR and the minimum mean square error of the estimator, the mutual information can be bounded as well for arbitrary source distributions in Gaussian noise.

AB - We derive a method to bound the mutual information between a noisy and noiseless measurement exploiting the I-MMSE estimation and information theory connection. Modeling the source distribution as a Gaussian mixture model, a closed form expression for upper and lower bounds of the minimum mean square error is found using recent results. Using the connection between rate of information relative to SNR and the minimum mean square error of the estimator, the mutual information can be bounded as well for arbitrary source distributions in Gaussian noise.

KW - Bounds

KW - Estimation Information

KW - Gaussian Mixture Models

KW - I-MMSE

UR - http://www.scopus.com/inward/record.url?scp=84992409183&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84992409183&partnerID=8YFLogxK

U2 - 10.1109/CISS.2016.7460514

DO - 10.1109/CISS.2016.7460514

M3 - Conference contribution

AN - SCOPUS:84992409183

SP - 274

EP - 279

BT - 2016 50th Annual Conference on Information Systems and Sciences, CISS 2016

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 50th Annual Conference on Information Systems and Sciences, CISS 2016

Y2 - 16 March 2016 through 18 March 2016

ER -

Estimation information bounds using the I-MMSE formula and Gaussian mixture models

Abstract

Other

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this