Improved finite-sample estimate of a nonparametric f-divergence

Prad Kadambi, Alan Wisler, Visar Berisha

Research output: Chapter in Book/Report/Conference proceedingConference contribution


Information divergence functions allow us to measure distances between probability density functions. We focus on the case where we only have data from the two distributions and have no knowledge of the underlying models from which the data is sampled. In this scenario, we consider an f-divergence for which there exists an asymptotically consistent, nonparametric estimator based on minimum spanning trees, the Dp divergence. Nonparametric estimators are known to have slow convergence rates in higher dimensions (d > 4), resulting in a large bias for small datasets. Based on experimental validation, we conjecture that the original estimator follows a power law convergence model and introduce a new estimator based on a bootstrap sampling scheme that results in a reduced bias. Experiments on real and artificial data show that the new estimator results in improved estimates of the Dp divergence when compared against the original estimator.

Original languageEnglish (US)
Title of host publicationConference Record of 51st Asilomar Conference on Signals, Systems and Computers, ACSSC 2017
EditorsMichael B. Matthews
PublisherInstitute of Electrical and Electronics Engineers Inc.
Number of pages5
ISBN (Electronic)9781538618233
StatePublished - Apr 10 2018
Event51st Asilomar Conference on Signals, Systems and Computers, ACSSC 2017 - Pacific Grove, United States
Duration: Oct 29 2017Nov 1 2017


Other51st Asilomar Conference on Signals, Systems and Computers, ACSSC 2017
Country/TerritoryUnited States
CityPacific Grove


  • asymptotic estimator
  • bootstrap estimator
  • minimal graphs
  • nonparametric f-Divergence

ASJC Scopus subject areas

  • Control and Optimization
  • Computer Networks and Communications
  • Hardware and Architecture
  • Signal Processing
  • Biomedical Engineering
  • Instrumentation


Dive into the research topics of 'Improved finite-sample estimate of a nonparametric f-divergence'. Together they form a unique fingerprint.

Cite this