Abstract
Multiple kernel learning (MKL) allows the practitioner to optimize over linear combinations of kernels and shows good performance in many applications. However, many MKL algorithms require very high computational costs in real world applications. In this study, we present a framework which uses multiple kernel SVM classifiers as the base learners for stacked generalization, a general method of using a high-level model to combine lower-level models, to achieve greater computational efficiency. The experimental results show that our MKL-based stacked generalization algorithm combines advantages from both MKL and stacked generalization. Compared to other general ensemble methods tested in this paper, this method achieves greater performance on predictive accuracy.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Wolpert, D.H.: Stacked Generalization. Neural Networks 5, 241–259 (1992)
Kai, M.T., Ian Witten, H.: Issues in Stacked Generalization. Journal of Artificial Intelligence Research 10, 271–289 (1999)
Bach, F.R., Lanckriet, G.R.G., Jordan, M.I.: Multiple kernel learning, conic duality, and the SMO algorithm. In: Proceedings of the 21st International Conference on Machine Learning (2004)
Vapnik, V.: The Nature of Statistical Learning Theory. Springer (1999)
Scholkopf, B., Smola, A., Muller, K.: Kernel principal component analysis. Advances in Kernel Methods: Support Vector Learning, 327–352 (1999)
Shawe-Taylor, J., Cristianini, N.: Kernel Methods for Pattern Analysis. Cambridge University Press (2004)
Frank, A., Asuncion, A.: UCI Machine Learning Repository. University of California, School of Information and Computer Science, Irvine, CA (2010), http://archive.ics.uci.edu/ml
Gehler, P.V., Nowozin, S.: Infinite kernel learning. Technical report, Max Planck Institute for Biological Cybernetics (2008)
Lanckriet, G., Cristianini, N., Bartlett, P., Ghaoui, L.E., Jordan, M.: Learning the kernel matrix with semidefinite programming. Journal of Machine Learning Research 5, 27–72 (2004)
Bach, F., Lanckriet, G., Jordan, M.: Multiple kernel learning, conic duality, and the smo algorithm. In: Proceedings of the 21st International Conference on Machine Learning, pp. 41–48 (2004)
Sonnenburg, S., Raetsch, G., Schaefer, C., Scholkopf, B.: Large scale multiple kernel learning. Journal of Machine Learning Research 7, 1531–1565 (2006)
Breiman, L.: Bagging predictors. Machine Learning 24(2), 123–140 (1996)
Ho, T.K.: The random subspace method for constructing decision forests. IEEE Transactions on Pattern Analysis and Machine Intelligence 20(8), 832–844 (1998)
Schapire, R.E.: The strength of weak learnability. Machine Learning 5, 197–227 (1990)
Wolpert, D.H., Macready, W.G.: Combining stacking with bagging to improve a learning algorithm. Technical Report SFI-TR-96-03-123, Santa Fe Institute, Santa Fe, New Mexico (1996)
Kai, M.T., Witten, H.I.: Stacking Bagged and Dagged Models. In: ICML, pp. 367–375 (1997)
Valentini, G., Dietterich, T.G.: Bias-Variance Analysis of Support Vector Machines for the Development of SVM-Based Ensemble Methods. Journal of Machine Learning Research 5, 725–775 (2004)
Kim, H.C., Pang, S., Je, H.M., Kim, D., Bang, S.Y.: Pattern Classification Using Support Vector Machine Ensemble. In: Proceedings of the International Conference on Pattern Recognition, vol. 2, pp. 20160–20163. IEEE (2002)
Valentini, G., Dietterich, T.G.: Low Bias Bagged Support Vector Machines. In: Fawcett, T., Mishra, N. (eds.) Proceedings of the Twentieth International Conference, Machine Learning, pp. 752–759. AAAI Press, Washington (2003)
Evgeniou, T., Perez-Breva, L., Pontil, M., Poggio, T.: Bounds on the Generalization Performance of Kernel Machine Ensembles. In: Langley, P. (ed.) Proc. of the Seventeenth International Conference on Machine Learning, pp. 271–278. Morgan Kaufmann (2000)
Friedman, J.H., Hastie, T., Tibshirani, R.J.: Additive logistic regression: a statistical view of boosting. Technical report, Stanford University, Department of Statistics (1998)
Cortes, C., Mohri, M., Rostamizadeh, A.: L2 regularization for learning kernels. In: Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence (2009)
Japkowicz, N., Shah, M.: Evaluating Learning Algorithms: A Classification Perspective. Cambridge University Press (2011)
Joachims, T., Cristianini, N., Shawe-Taylor, J.: Composite kernels for hypertext categorisation. In: Proceedings of the 18th International Conference on Machine Learning (2001)
Alham, N.K., Li, M., Liu, Y.: A distributed SVM ensemble for image: Classification and annotation. In: Proceedings of the 9th International Conference on Fuzzy Systems and Knowledge Discovery, pp. 1581–1584. IEEE, Piscataway (2012)
Chang, E.Y., Zhu, K., Wang, H.: PSVM: Parallelizing support vector machines on distributed computers. Adv. Neural Inf. Process Syst. 20, 1–8 (2007)
Chen, Z.Y., Fan, Z.P.: Parallel multiple kernel learning: A hybrid alternating direction method of multipliers. Knowledge and Information Systems (2013)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this paper
Cite this paper
Wang, X., Liu, X., Japkowicz, N., Matwin, S. (2014). Ensemble of Multiple Kernel SVM Classifiers. In: Sokolova, M., van Beek, P. (eds) Advances in Artificial Intelligence. Canadian AI 2014. Lecture Notes in Computer Science(), vol 8436. Springer, Cham. https://doi.org/10.1007/978-3-319-06483-3_21
Download citation
DOI: https://doi.org/10.1007/978-3-319-06483-3_21
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-06482-6
Online ISBN: 978-3-319-06483-3
eBook Packages: Computer ScienceComputer Science (R0)