Learning to Predict the Leave-One-Out Error of Kernel Based Classifiers
We propose an algorithm to predict the leave-one-out (LOO) error for kernel based classifiers. To achieve this goal with computational efficiency, we cast the LOO error approximation task into a classification problem. This means that we need to learn a classification of whether or not a given training sample - if left out of the data set - would be misclassified. For this learning task, simple data dependent features are proposed, inspired by geometrical intuition. Our approach allows to reliably select a good model as demonstrated in simulations on Support Vector and Linear Programming Machines. Comparisons to existing learning theoretical bounds, e.g. the span bound, are given for various model selection scenarios.
KeywordsTraining Sample Theoretical Bound Multiclass Problem Meta Learning Dual Weight
Unable to display preview. Download preview PDF.
- 2.O. Chapelle and V.N. Vapnik. Choosing kernel parameters for support vector machines. Personal communication, mar 2000.Google Scholar
- 3.T.S. Jaakkola and D. Haussler. Probabilistic kernel regression models. In Proceedings of the 1999 Conference on AI and Statistics, 1999.Google Scholar
- 4.K.-R. Müller, S. Mika, G. Rätsch, K. Tsuda, and B. Schölkopf. An introduction to kernel-based learning algorithms. IEEE Transactions on Neural Networks, 2001. in Press.Google Scholar
- 5.M. Opper and O. Winther. Gaussian processes and SVM: Mean field and leave-one-out. In A.J. Smola, P.L. Bartlett, B. Schölkopf, and D. Schuurmans, editors, Advances in Large Margin Classifiers, pages 311–326, Cambridge, MA, 2000. MIT Press.Google Scholar
- 8.V.N. Vapnik and O. Chapelle. Bounds on error expectation for support vector machines. Neural Computation, 12(9):2013–2036, September 2000.Google Scholar
- 9.G. Whaba, Y. Lin, and H. Zhang. Generalized approximate cross-validation for support-vector-machines: another way to look at margin-like quantities. Technical Report TR-1006, Dept. of Statistics, University of Wisconsin, April 1999.Google Scholar