Abstract
Based on the feature map principle, Sparse Kernel Ridge Regression (SKRR) model is proposed. SKRR obtains the sparseness by backward deletion feature selection procedure that recursively removes the feature with the smallest leave-one-out score until the stop criterion is satisfied. Besides good generalization performance, the most compelling property of SKRR is rather sparse, and moreover, the kernel function needs not to be positive definite. Experiments on synthetic and benchmark data sets validate the feasibility and validity of SKRR.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Vapnik, V.: The Nature of Statistical Learning Theory. Springer, New York (1995)
Vapnik, V.: Statistical Learning Theory. Wiley-Interscience Publication, New York (1998)
Tipping, M.E.: The Relevance Vector Machine. In: Solla, S., Leen, T., Müller, K.-R. (eds.) Proc. Advances in Neural Information Processing Systems, vol. 12, pp. 652–658. MIT Press, Cambridge (2001)
Neal, R.: Bayesian Learning for Neural Networks. Springer, New York (1996)
Ripley, R.: Pattern Recognition and Neural Networks. Cambridge Univ. Press, Cambridge (1996)
Hoerl, A., Kennard, R.: Ridge Regression: Biased Estimation for Nonorthogonal Problems. Technometrics 12, 55–67 (1970)
Saunder, C., Gammerman, A.: Ridge regression learning algorithm in dual variables. In: Shavlik, J. (ed.) Machine learning Proceedings of the 15th International Conference, Morgan Kaufmann, San Francisco (1998)
Stoer, J., Bulirsch, R.: Introduction to Numerical Analysis, 2nd edn. Springer, New York (1993)
Bo, L.F., Wang, L., Jiao, L.C.: Sparse Gaussian processes using backward elimination. In: Wang, J., Yi, Z., Żurada, J.M., Lu, B.-L., Yin, H. (eds.) ISNN 2006. LNCS, vol. 3971, pp. 1083–1088. Springer, Heidelberg (2006)
Bo, L.F., Wang, L., Jiao, L.C.: Feature scaling for kernel Fisher discriminant analysis using leave-one-out cross validation. Neural Computation 18(4), 961–978 (2006)
Allen, D.M.: The relationship between variable selection and prediction. Technometrics 16, 125–127 (1974)
Gavin, C.C.: Efficient leave-one-out cross-validation of kernel fisher discriminant classifiers. Pattern Recognition 36(11), 2585–2592 (2003)
Michie, D., Spiegelhalter, D.J., Taylor, C.C.: Machine Learning, Neural and Statistical Classification. Prentice-Hall, Englewood Cliffs (1994), Data available at anonymous http://ftp.ncc.up.pt/pub/statlog/
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Wang, L., Bo, L., Jiao, L. (2006). Sparse Kernel Ridge Regression Using Backward Deletion. In: Yang, Q., Webb, G. (eds) PRICAI 2006: Trends in Artificial Intelligence. PRICAI 2006. Lecture Notes in Computer Science(), vol 4099. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-36668-3_40
Download citation
DOI: https://doi.org/10.1007/978-3-540-36668-3_40
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-36667-6
Online ISBN: 978-3-540-36668-3
eBook Packages: Computer ScienceComputer Science (R0)