Abstract
In this paper we present a novel approach to combining multiple kernels where the kernels are computed from different information channels. In contrast to traditional methods that learn a linear combination of n kernels of size m ×m, resulting in m coefficients in the trained classifier, we propose a method that can learn n ×m coefficients. This allows to assign different importance to the information channel per example rather than per kernel. We analyse the proposed kernel combination in empirical feature space and provide its geometrical interpretation. We validate the approach on both UCI datasets and an object recognition dataset, and demonstrate that it leads to classification improvements.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Scholkopf, B., Smola, A.: Learning with Kernels. MIT Press, Cambridge (2002)
Shawe-Taylor, J., Cristianini, N.: Kernel Methods for Pattern Analysis. Cambridge University Press, Cambridge (2004)
Vapnik, V.: The Nature of Statistical Learning Theory. Springer, Heidelberg (1999)
Mika, S.: Kernel fisher discriminants. PhD Thesis, University of Technology, Berlin, Germany (2002)
Baudat, G., Anouar, F.: Generalized discriminant analysis using a kernel approach. Neural Computation 12, 2385–2404 (2000)
Scholkopf, B., Smola, A., Muller, K.: Kernel principal component analysis. In: Advances in Kernel Methods: Support Vector Learning, pp. 327–352 (1999)
Lanckriet, G., Cristianini, N., Bartlett, P., Ghaoui, L.E., Jordan, M.: Learning the kernel matrix with semidefinite programming. JMLR 5, 27–72 (2004)
Bach, F., Lanckriet, G.: Multiple kernel learning, conic duality, and the smo algorithm. In: ICML (2004)
Sonnenburg, S., Ratsch, G., Schafer, C., Scholkopf, B.: Large scale multiple kernel learning. JMLR 7, 1531–1565 (2006)
Rakotomamonjy, A., Bach, F., Grandvalet, Y., Canu, S.: Simplemkl. JMLR 9, 2491–2521 (2008)
Zien, A., Ong, C.: Multiclass multiple kernel learning. In: ICML, pp. 1191–1198 (2007)
Ji, S., Sun, L., Jin, R., Ye, J.: Multilabel multiple kernel learning. In: NIPS (2008)
Kim, S., Magnani, A., Boyd, S.: Optimal kernel selection in kernel fisher discriminant analysis. In: ICML (2006)
Ye, J., Ji, S., Chen, J.: Multi-class discriminant kernel learning via convex programming. JMLR 9, 719–758 (2008)
Yan, F., Kittler, J., Mikolajczyk, K., Tahir, A.: Non-sparse multiple kernel learning for fisher discriminant analysis. In: ICDM (2009)
Kloft, M., Brefeld, U., Laskov, P., Sonnenburg, S.: Non-sparse multiple kernel learning. In: NIPS Workshop on Kernel Learning: Automatic Selection of Optimal Kernels (2008)
Kloft, M., Brefeld, U., Sonnenburg, S., Zien, A.: Efficient and accurate lp-norm mkl. In: NIPS (2009)
Cortes, C., Mohri, M., Rostamizadeh, A.: L2 regularization for learning kernels. In: UAI (2009)
Scholkopf, B., Mika, S., Burges, C., Knirsch, P., Muller, K., Ratsch, G., Smola, A.: Input space versus feature space in kerenl-based methods. IEEE Transactions on Neural Networks 10(5), 1000–1017 (1999)
Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning, 2nd edn. Springer, Heidelberg (2002)
Newman, D., Hettich, S., Blake, C., Merz, C.: Uci repository of machine learning databases (1998), http://www.ics.uci.edu/~mlearn/MLRepository.html
Cai, D., He, X., Han, J.: Efficient kernel discriminant analysis via spectral regression. In: ICDM (2007)
Everingham, M., Gool, L.V., Williams, C.K.I., Winn, J., Zisserman, A.: The PASCAL Visual Object Classes Challenge 2008 (VOC2008) Results (2008), http://www.pascal-network.org/challenges/VOC/voc2008/workshop/index.html
Snoek, C., Worring, M., Gemert, J., Geusebroek, J., Smeulders, A.: The challenge problem for automated detection of 101 semantic concepts in multimedia. In: ACM Multimedia Conference, pp. 421–430 (2006)
Mikolajczyk, K., Schmid, C.: A performance evaluation of local descriptors. PAMI 27(10), 1615–1630 (2005)
Gemert, J., Geusebroek, J., Veenman, C., Smeulders, A.: Kernel codebooks for scene categorization. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part III. LNCS, vol. 5304, pp. 696–709. Springer, Heidelberg (2008)
Sande, K., Gevers, T., Snoek, C.: Evaluation of color descriptors for object and scene recognition. In: CVPR (2008)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Yan, F., Mikolajczyk, K., Kittler, J., Tahir, M.A. (2010). Combining Multiple Kernels by Augmenting the Kernel Matrix. In: El Gayar, N., Kittler, J., Roli, F. (eds) Multiple Classifier Systems. MCS 2010. Lecture Notes in Computer Science, vol 5997. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-12127-2_18
Download citation
DOI: https://doi.org/10.1007/978-3-642-12127-2_18
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-12126-5
Online ISBN: 978-3-642-12127-2
eBook Packages: Computer ScienceComputer Science (R0)