Challenges in benchmarking metagenomic profilers

Zheng Sun, Shi Huang, Meng Zhang, Qiyun Zhu, Niina Haiminen, Anna Paola Carrieri, Yoshiki Vázquez-Baeza, Laxmi Parida, Ho Cheol Kim, Rob Knight, Yang Yu Liu

Research output: Contribution to journalArticlepeer-review

41 Scopus citations


Accurate microbial identification and abundance estimation are crucial for metagenomics analysis. Various methods for classification of metagenomic data and estimation of taxonomic profiles, broadly referred to as metagenomic profilers, have been developed. Nevertheless, benchmarking of metagenomic profilers remains challenging because some tools are designed to report relative sequence abundance while others report relative taxonomic abundance. Here we show how misleading conclusions can be drawn by neglecting this distinction between relative abundance types when benchmarking metagenomic profilers. Moreover, we show compelling evidence that interchanging sequence abundance and taxonomic abundance will influence both per-sample summary statistics and cross-sample comparisons. We suggest that the microbiome research community pay attention to potentially misleading biological conclusions arising from this issue when benchmarking metagenomic profilers, by carefully considering the type of abundance data that were analyzed and interpreted and clearly stating the strategy used for metagenomic profiling.

Original languageEnglish (US)
Pages (from-to)618-626
Number of pages9
JournalNature Methods
Issue number6
StatePublished - Jun 2021
Externally publishedYes

ASJC Scopus subject areas

  • Biotechnology
  • Biochemistry
  • Molecular Biology
  • Cell Biology


Dive into the research topics of 'Challenges in benchmarking metagenomic profilers'. Together they form a unique fingerprint.

Cite this