TY - GEN
T1 - Treed-Gaussian Processes with Support Vector Machines as Nodes for Nonstationary Bayesian Optimization
AU - Candelieri, Antonio
AU - Pedrielli, Giulia
N1 - Funding Information:
We greatly acknowledge the DEMS Data Science Lab, Department of Economics Management and Statistics (DEMS), University of Milano-Bicocca, for supporting this work by providing computational resources. The research presented in this work was partially supported under grants NSF#1829238-#2007861-#2026860.
Publisher Copyright:
© 2021 IEEE.
PY - 2021
Y1 - 2021
N2 - A large family of black box methods rely on surrogates of the unknown, possibly non linear non convex reward function. While it is common to assume stationarity of the reward, many real-world problems satisfy this assumption only locally, hindering the spread application of such methods. This paper proposes a novel nonstationary regression model combining Decision Trees and Support Vector Machine (SVM) classification for a hierarchical non-axis-aligned partition of the input space. Gaussian Process (GP) regression is performed within each identified subregion. The resulting nonstationary regression model is the Treed Gaussian process with Support Vector Machine (SVMTGP), and we investigate the sampling efficiency from using our a model within a Bayesian optimization (BO) context. Empirically, we show how the resulting algorithm, SVMTGP-BO never underperforms BO when this is applied to an homogeneous Gaussian process, while it shows always better performance compared to the homogeneous model with nonlinear functions with complex landscapes.
AB - A large family of black box methods rely on surrogates of the unknown, possibly non linear non convex reward function. While it is common to assume stationarity of the reward, many real-world problems satisfy this assumption only locally, hindering the spread application of such methods. This paper proposes a novel nonstationary regression model combining Decision Trees and Support Vector Machine (SVM) classification for a hierarchical non-axis-aligned partition of the input space. Gaussian Process (GP) regression is performed within each identified subregion. The resulting nonstationary regression model is the Treed Gaussian process with Support Vector Machine (SVMTGP), and we investigate the sampling efficiency from using our a model within a Bayesian optimization (BO) context. Empirically, we show how the resulting algorithm, SVMTGP-BO never underperforms BO when this is applied to an homogeneous Gaussian process, while it shows always better performance compared to the homogeneous model with nonlinear functions with complex landscapes.
UR - http://www.scopus.com/inward/record.url?scp=85126140928&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85126140928&partnerID=8YFLogxK
U2 - 10.1109/WSC52266.2021.9715514
DO - 10.1109/WSC52266.2021.9715514
M3 - Conference contribution
AN - SCOPUS:85126140928
T3 - Proceedings - Winter Simulation Conference
BT - 2021 Winter Simulation Conference, WSC 2021
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 2021 Winter Simulation Conference, WSC 2021
Y2 - 12 December 2021 through 15 December 2021
ER -