TY - GEN
T1 - Neural Network-based Estimation of the MMSE
AU - Diaz, Mario
AU - Kairouz, Peter
AU - Liao, Jiachun
AU - Sankar, Lalitha
N1 - Funding Information:
ACKNOWLEDGEMENTS The work of L. Sankar and J. Liao is supported in part by NSF grants CIF-1901243, CIF-1815361, and CIF-2007688. The work of M. Diaz was supported in part by the Programa de Apoyo a Proyectos de Investigación e Innovación Tecnológica (PAPIIT) under grant IA101021.
Publisher Copyright:
© 2021 IEEE.
PY - 2021/7/12
Y1 - 2021/7/12
N2 - The minimum mean-square error (MMSE) achievable by optimal estimation of a random variable S given another random variable T is of much interest in a variety of statistical contexts. Motivated by a growing interest in auditing machine learning models for unintended information leakage, we propose a neural network-based estimator of this MMSE. We derive a lower bound for the MMSE based on the proposed estimator and the Barron constant associated with the conditional expectation of S given T. Since the latter is typically unknown in practice, we derive a general bound for the Barron constant that produces order optimal estimates for canonical distribution models.
AB - The minimum mean-square error (MMSE) achievable by optimal estimation of a random variable S given another random variable T is of much interest in a variety of statistical contexts. Motivated by a growing interest in auditing machine learning models for unintended information leakage, we propose a neural network-based estimator of this MMSE. We derive a lower bound for the MMSE based on the proposed estimator and the Barron constant associated with the conditional expectation of S given T. Since the latter is typically unknown in practice, we derive a general bound for the Barron constant that produces order optimal estimates for canonical distribution models.
UR - http://www.scopus.com/inward/record.url?scp=85114478499&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85114478499&partnerID=8YFLogxK
U2 - 10.1109/ISIT45174.2021.9518063
DO - 10.1109/ISIT45174.2021.9518063
M3 - Conference contribution
AN - SCOPUS:85114478499
T3 - IEEE International Symposium on Information Theory - Proceedings
SP - 1023
EP - 1028
BT - 2021 IEEE International Symposium on Information Theory, ISIT 2021 - Proceedings
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 2021 IEEE International Symposium on Information Theory, ISIT 2021
Y2 - 12 July 2021 through 20 July 2021
ER -