Abstract
In selecting input variables by block addition and block deletion (BABD), multiple input variables are added and then deleted, keeping the cross-validation error below that using all the input variables. The major problem of this method is that selection time becomes large as the number of input variables increases. To alleviate this problem, in this paper, we propose incremental block addition and block deletion of input variables. In this method, for an initial subset of input variables we select input variables by BABD. Then in the incremental step, we add some input variables that are not added before to the current selected input variables and iterate BABD. To guarantee that the cross-validation error decreases monotonically by incremental BABD, we undo incremental BABD if the obtained cross-validation error rate is worse than that at the previous incremental step. We evaluate incremental BABD using some benchmark data sets and show that by incremental BABD, input variable selection is speeded up with the approximation error comparable to that by batch BABD.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Vapnik, V.N.: Statistical Learning Theory. John Wiley & Sons (1998)
Suykens, J.A.K.: Least squares support vector machines for classification and nonlinear modeling. Neural Network World 10(1–2), 29–47 (2000)
Abe, S.: Support Vector Machines for Pattern Classification. Springer (2005)
Fung, G.M., Mangasarian, O.L.: A feature selection Newton method for support vector machine classification. Computational Optimization and Applications 28(2), 185–202 (2004)
Stearns, S.D.: On selecting features for pattern classifiers. In: Proc. Third International Conference on Pattern Recognition, pp. 71–75 (1976)
Pudil, P., Novovičová, J., Kittler, J.: Floating search methods in feature selection. Pattern Recognition Letters 15(11), 1119–1125 (1994)
Zhang, T.: Adaptive forward-backward greedy algorithm for sparse learning with linear models. In: Proc. NIPS 1921, pp. 1921–1928. MIT Press (2009)
Liu, H., Setiono, R.: Incremental feature selection. Applied Intelligence 9(3), 217–230 (1998)
Perkins, S., Lacker, K., Theiler, J.: Grafting: Fast, incremental feature selection by gradient descent in function space. Journal of Machine Learning Research 3, 1333–1356 (2003)
Ruiz, R., Riquelme, J.C., Aguilar-Ruiz, J.S.: Incremental wrapper-based gene selection from microarray data for cancer classification. Pattern Recognition 39(12), 2383–2392 (2006)
Bermejo, P., Gamez, J.A., Puerta, J.M.: Speeding up incremental wrapper feature subset selection with naive Bayes classifier. Knowledge-Based Systems 55, 140–147 (2014)
Nagatani, T., Ozawa, S., Abe, S.: Fast variable selection by block addition and block deletion. Journal of Intelligent Learning Systems and Applications 2(4), 200–211 (2010)
Nagatani, T., Abe, S.: Feature selection by block addition and block deletion. In: Mana, N., Schwenker, F., Trentin, E. (eds.) ANNPR 2012. LNCS (LNAI), vol. 7477, pp. 48–59. Springer, Heidelberg (2012)
Asuncion, A., Newman, D.J.: UCI machine learning repository (2007), http://www.ics.uci.edu/~mlearn/MLRepository.html
Milano Chemometrics and QSAR Research Group , http://michem.disat.unimib.it/chm/download/download.htm
UCL Machine Learning Group, http://mlg.info.ucl.ac.be/index.php?page=DataBases
Hedenfalk, I., et al.: Gene-expression profiles in hereditary breast cancer. The New England Journal of Medicine 344(8), 539–548 (2001)
Golub, T.R., et al.: Molecular classification of cancer: Class discovery and class prediction by gene expression monitoring. Science 286, 531–537 (1999)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this paper
Cite this paper
Abe, S. (2014). Incremental Input Variable Selection by Block Addition and Block Deletion. In: Wermter, S., et al. Artificial Neural Networks and Machine Learning – ICANN 2014. ICANN 2014. Lecture Notes in Computer Science, vol 8681. Springer, Cham. https://doi.org/10.1007/978-3-319-11179-7_69
Download citation
DOI: https://doi.org/10.1007/978-3-319-11179-7_69
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-11178-0
Online ISBN: 978-3-319-11179-7
eBook Packages: Computer ScienceComputer Science (R0)