A System-driven taxonomy of attacks and defenses in adversarial machine learning

Research output: Contribution to journalArticlepeer-review

38 Scopus citations


Machine Learning (ML) algorithms, specifically supervised learning, are widely used in modern real-world applications, which utilize Computational Intelligence (CI) as their core technology, such as autonomous vehicles, assistive robots, and biometric systems. Attacks that cause misclassifications or mispredictions can lead to erroneous decisions resulting in unreliable operations. Designing robust ML with the ability to provide reliable results in the presence of such attacks has become a top priority in the field of adversarial machine learning. An essential characteristic for rapid development of robust ML is an arms race between attack and defense strategists. However, an important prerequisite for the arms race is access to a well-defined system model so that experiments can be repeated by independent researchers. This article proposes a fine-grained system-driven taxonomy to specify ML applications and adversarial system models in an unambiguous manner such that independent researchers can replicate experiments and escalate the arms race to develop more evolved and robust ML applications. The article provides taxonomies for: 1) the dataset, 2) the ML architecture, 3) the adversary's knowledge, capability, and goal, 4) adversary's strategy, and 5) the defense response. In addition, the relationships among these models and taxonomies are analyzed by proposing an adversarial machine learning cycle. The provided models and taxonomies are merged to form a comprehensive system-driven taxonomy, which represents the arms race between the ML applications and adversaries in recent years. The taxonomies encode best practices in the field and help evaluate and compare the contributions of research works and reveals gaps in the field.

Original languageEnglish (US)
Article number9099439
Pages (from-to)450-467
Number of pages18
JournalIEEE Transactions on Emerging Topics in Computational Intelligence
Issue number4
StatePublished - Aug 2020


  • Computational intelligence (CI)
  • adversarial machine learning
  • attack model
  • defense model
  • supervised learning

ASJC Scopus subject areas

  • Computer Science Applications
  • Control and Optimization
  • Computational Mathematics
  • Artificial Intelligence


Dive into the research topics of 'A System-driven taxonomy of attacks and defenses in adversarial machine learning'. Together they form a unique fingerprint.

Cite this