MLPerf: An industry standard benchmark suite for machine learning performance

Peter Mattson; Hanlin Tang; Gu Yeon Wei; Carole Jean Wu; Vijay Janapa Reddi; Christine Cheng; Cody Coleman; Greg Diamos; David Kanter; Paulius Micikevicius; David Patterson; Guenther Schmuelling

doi:10.1109/MM.2020.2974843

MLPerf: An industry standard benchmark suite for machine learning performance

Peter Mattson, Hanlin Tang, Gu Yeon Wei, Carole Jean Wu, Vijay Janapa Reddi, Christine Cheng, Cody Coleman, Greg Diamos, David Kanter, Paulius Micikevicius, David Patterson, Guenther Schmuelling

Research output: Contribution to journal › Article › peer-review

86 Scopus citations

Abstract

In this article, we describe the design choices behind MLPerf, a machine learning performance benchmark that has become an industry standard. The first two rounds of the MLPerf Training benchmark helped drive improvements to software-stack performance and scalability, showing a 1.3× speedup in the top 16-chip results despite higher quality targets and a 5.5× increase in system scale. The first round of MLPerf Inference received over 500 benchmark results from 14 different organizations, showing growing adoption.

Original language	English (US)
Article number	9001257
Pages (from-to)	8-16
Number of pages	9
Journal	IEEE Micro
Volume	40
Issue number	2
DOIs	https://doi.org/10.1109/MM.2020.2974843
State	Published - Mar 1 2020
Externally published	Yes

ASJC Scopus subject areas

Software
Hardware and Architecture
Electrical and Electronic Engineering

Access to Document

10.1109/MM.2020.2974843

Cite this

@article{c6ef7cc883b8493095542deb48526d4a,

title = "MLPerf: An industry standard benchmark suite for machine learning performance",

abstract = "In this article, we describe the design choices behind MLPerf, a machine learning performance benchmark that has become an industry standard. The first two rounds of the MLPerf Training benchmark helped drive improvements to software-stack performance and scalability, showing a 1.3× speedup in the top 16-chip results despite higher quality targets and a 5.5× increase in system scale. The first round of MLPerf Inference received over 500 benchmark results from 14 different organizations, showing growing adoption.",

author = "Peter Mattson and Hanlin Tang and Wei, {Gu Yeon} and Wu, {Carole Jean} and Reddi, {Vijay Janapa} and Christine Cheng and Cody Coleman and Greg Diamos and David Kanter and Paulius Micikevicius and David Patterson and Guenther Schmuelling",

note = "Publisher Copyright: {\textcopyright} 1981-2012 IEEE.",

year = "2020",

month = mar,

day = "1",

doi = "10.1109/MM.2020.2974843",

language = "English (US)",

volume = "40",

pages = "8--16",

journal = "IEEE Micro",

issn = "0272-1732",

publisher = "IEEE Computer Society",

number = "2",

}

TY - JOUR

T1 - MLPerf

T2 - An industry standard benchmark suite for machine learning performance

AU - Mattson, Peter

AU - Tang, Hanlin

AU - Wei, Gu Yeon

AU - Wu, Carole Jean

AU - Reddi, Vijay Janapa

AU - Cheng, Christine

AU - Coleman, Cody

AU - Diamos, Greg

AU - Kanter, David

AU - Micikevicius, Paulius

AU - Patterson, David

AU - Schmuelling, Guenther

PY - 2020/3/1

Y1 - 2020/3/1

N2 - In this article, we describe the design choices behind MLPerf, a machine learning performance benchmark that has become an industry standard. The first two rounds of the MLPerf Training benchmark helped drive improvements to software-stack performance and scalability, showing a 1.3× speedup in the top 16-chip results despite higher quality targets and a 5.5× increase in system scale. The first round of MLPerf Inference received over 500 benchmark results from 14 different organizations, showing growing adoption.

AB - In this article, we describe the design choices behind MLPerf, a machine learning performance benchmark that has become an industry standard. The first two rounds of the MLPerf Training benchmark helped drive improvements to software-stack performance and scalability, showing a 1.3× speedup in the top 16-chip results despite higher quality targets and a 5.5× increase in system scale. The first round of MLPerf Inference received over 500 benchmark results from 14 different organizations, showing growing adoption.

UR - http://www.scopus.com/inward/record.url?scp=85079606889&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85079606889&partnerID=8YFLogxK

U2 - 10.1109/MM.2020.2974843

DO - 10.1109/MM.2020.2974843

M3 - Article

AN - SCOPUS:85079606889

SN - 0272-1732

VL - 40

SP - 8

EP - 16

JO - IEEE Micro

JF - IEEE Micro

IS - 2

M1 - 9001257

ER -

MLPerf: An industry standard benchmark suite for machine learning performance

Abstract

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this