TY - GEN
T1 - Feature selection via regularized trees
AU - Deng, Houtao
AU - Runger, George
N1 - Copyright:
Copyright 2012 Elsevier B.V., All rights reserved.
PY - 2012
Y1 - 2012
N2 - We propose a tree regularization framework, which enables many tree models to perform feature selection efficiently. The key idea of the regularization framework is to penalize selecting a new feature for splitting when its gain (e.g. information gain) is similar to the features used in previous splits. The regularization framework is applied on random forest and boosted trees here, and can be easily applied to other tree models. Experimental studies show that the regularized trees can select high-quality feature subsets with regard to both strong and weak classifiers. Because tree models can naturally deal with categorical and numerical variables, missing values, different scales between variables, interactions and nonlinearities etc., the tree regularization framework provides an effective and efficient feature selection solution for many practical problems.
AB - We propose a tree regularization framework, which enables many tree models to perform feature selection efficiently. The key idea of the regularization framework is to penalize selecting a new feature for splitting when its gain (e.g. information gain) is similar to the features used in previous splits. The regularization framework is applied on random forest and boosted trees here, and can be easily applied to other tree models. Experimental studies show that the regularized trees can select high-quality feature subsets with regard to both strong and weak classifiers. Because tree models can naturally deal with categorical and numerical variables, missing values, different scales between variables, interactions and nonlinearities etc., the tree regularization framework provides an effective and efficient feature selection solution for many practical problems.
KW - RBoost
KW - RRF
KW - regularized boosted trees
KW - regularized random forest
KW - tree regularization
UR - http://www.scopus.com/inward/record.url?scp=84865067900&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=84865067900&partnerID=8YFLogxK
U2 - 10.1109/IJCNN.2012.6252640
DO - 10.1109/IJCNN.2012.6252640
M3 - Conference contribution
AN - SCOPUS:84865067900
SN - 9781467314909
T3 - Proceedings of the International Joint Conference on Neural Networks
BT - 2012 International Joint Conference on Neural Networks, IJCNN 2012
T2 - 2012 Annual International Joint Conference on Neural Networks, IJCNN 2012, Part of the 2012 IEEE World Congress on Computational Intelligence, WCCI 2012
Y2 - 10 June 2012 through 15 June 2012
ER -