Treed-Gaussian Processes with Support Vector Machines as Nodes for Nonstationary Bayesian Optimization

Antonio Candelieri, Giulia Pedrielli

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Scopus citations


A large family of black box methods rely on surrogates of the unknown, possibly non linear non convex reward function. While it is common to assume stationarity of the reward, many real-world problems satisfy this assumption only locally, hindering the spread application of such methods. This paper proposes a novel nonstationary regression model combining Decision Trees and Support Vector Machine (SVM) classification for a hierarchical non-axis-aligned partition of the input space. Gaussian Process (GP) regression is performed within each identified subregion. The resulting nonstationary regression model is the Treed Gaussian process with Support Vector Machine (SVMTGP), and we investigate the sampling efficiency from using our a model within a Bayesian optimization (BO) context. Empirically, we show how the resulting algorithm, SVMTGP-BO never underperforms BO when this is applied to an homogeneous Gaussian process, while it shows always better performance compared to the homogeneous model with nonlinear functions with complex landscapes.

Original languageEnglish (US)
Title of host publication2021 Winter Simulation Conference, WSC 2021
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9781665433112
StatePublished - 2021
Event2021 Winter Simulation Conference, WSC 2021 - Phoenix, United States
Duration: Dec 12 2021Dec 15 2021

Publication series

NameProceedings - Winter Simulation Conference
ISSN (Print)0891-7736


Conference2021 Winter Simulation Conference, WSC 2021
Country/TerritoryUnited States

ASJC Scopus subject areas

  • Software
  • Modeling and Simulation
  • Computer Science Applications


Dive into the research topics of 'Treed-Gaussian Processes with Support Vector Machines as Nodes for Nonstationary Bayesian Optimization'. Together they form a unique fingerprint.

Cite this