A new correlation coefficient for comparing and aggregating non-strict and incomplete rankings

Yeawon Yoo, Adolfo R. Escobedo, J. Kyle Skolfield

Research output: Contribution to journalArticlepeer-review

23 Scopus citations


We introduce a correlation coefficient that is designed to deal with a variety of ranking formats including those containing non-strict (i.e., with-ties) and incomplete (i.e., unknown) preferences. The correlation coefficient is designed to enforce a neutral treatment of incompleteness whereby no assumptions are made about individual preferences involving unranked objects. The new measure, which can be regarded as a generalization of the seminal Kendall tau correlation coefficient, is proven to satisfy a set of metric-like axioms and to be equivalent to a recently developed ranking distance function associated with Kemeny aggregation. In an effort to further unify and enhance both robust ranking methodologies, this work proves the equivalence of an additional distance and correlation-coefficient pairing in the space of non-strict incomplete rankings. These connections induce new exact optimization methodologies: a specialized branch and bound algorithm and an exact integer programming formulation. Moreover, the bridging of these complementary theories reinforces the singular suitability of the featured correlation coefficient to solve the general consensus ranking problem. The latter premise is bolstered by an accompanying set of experiments on random instances, which are generated via a herein developed sampling technique connected with the classic Mallows distribution of ranking data. Associated experiments with the branch and bound algorithm demonstrate that, as data becomes noisier, the featured correlation coefficient yields relatively fewer alternative optimal solutions and that the aggregate rankings tend to be closer to an underlying ground truth shared by a majority.

Original languageEnglish (US)
Pages (from-to)1025-1041
Number of pages17
JournalEuropean Journal of Operational Research
Issue number3
StatePublished - Sep 16 2020


  • Correlation and distance functions
  • Group decisions and negotiations
  • Non-strict incomplete rankings
  • Robust ranking aggregation

ASJC Scopus subject areas

  • General Computer Science
  • Modeling and Simulation
  • Management Science and Operations Research
  • Information Systems and Management


Dive into the research topics of 'A new correlation coefficient for comparing and aggregating non-strict and incomplete rankings'. Together they form a unique fingerprint.

Cite this