Skip to main content
SpringerLink
Log in

Joint Learning of Distance Metric and Kernel Classifier via Multiple Kernel Learning

  • Conference paper
  • First Online:
Pattern Recognition (CCPR 2016)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 662))

Included in the following conference series:

  • 1771 Accesses

Abstract

Both multiple kernel learning (MKL) and support vector metric learning (SVML) were developed to adaptively learn kernel function from training data, and have been proved to be effective in many challenging applications. Actually, many MKL formulations are based on either the max-margin or radius-margin based principles, which in spirit is consistent with the optimization principle of the between/within class distances adopted in SVML. This motivates us to investigate their connection and develop a novel model for joint learning distance metric and kernel classifier. In this paper, we provide a new parameterization scheme for incorporating the squared Mahalanobis distance into the Gaussian RBF kernel, and formulate kernel learning into a GMKL framework. Moreover, radius information is also incorporated as the supplement for considering the within-class distance in the feature space. We demonstrate the effectiveness of the proposed algorithm on several benchmark datasets of varying sizes and difficulties. Experimental results show that the proposed algorithm achieves competitive classification accuracies with both state-of-the-art metric learning models and representative kernel learning models.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    http://www.cs.cornell.edu/~kilian/code/code.html.

  2. 2.

    http://www.cs.utexas.edu/~pjain/itml.

  3. 3.

    http://www.cse.wustl.edu/~xuzx/research/code/code.html.

  4. 4.

    http://research.microsoft.com/en-us/um/people/manik/code/GMKL/download.ht ml.

  5. 5.

    http://asi.insa-rouen.fr/enseignants/~arakoto/code/mklindex.html.

  6. 6.

    https://www.csie.ntu.edu.tw/~cjlin/libsvm/.

References

  1. Lampert, C.H.: Kernel methods in computer vision. Found. Trends Comput. Graph. Vis. 4(3), 193–285 (2009)

    Article  MathSciNet  Google Scholar 

  2. Vedaldi, A., Gulshan, V., Varma, M., Zisserman, A.: Multiple kernels for object detection. In: IEEE International Conference on Computer Vision (ICCV), pp. 606–613 (2009)

    Google Scholar 

  3. Chen, L., Chen, C., Lu, M.: A multiple-kernel fuzzy c-means algorithm for image segmentation. IEEE Trans. Syst. Man Cybern. B Cybern. 41(5), 1263–1274 (2011)

    Article  Google Scholar 

  4. Ye, J., Chen, K., Wu, T., Li, J., Zhao, Z., Patel, R., Bae, M., Janardan, R., Liu, H., Alexander, G., Reiman, E.: Heterogeneous data fusion for alzheimer’s disease study. In: ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1025–1033 (2008)

    Google Scholar 

  5. Chapelle, O., Vapnik, V., Bousquet, O., Mukherjee, S.: Choosing multiple parameters for support vector machines. Mach. Learn. 46(1), 131–159 (2002)

    Article  MATH  Google Scholar 

  6. Lanckriet, G., Cristianini, N., Bartlett, P., Chaoui, L.E., Jordan, M.: Learning the kernel matrix with semidefinite programming. J. Mach. Learn. Res. 5(1), 27–72 (2004)

    MathSciNet  MATH  Google Scholar 

  7. Bach, F., Lanckriet, G., Jordan, M.: Multiple kernel learning, conic duality, and the SMO algorithm. In: International Conference on Machine Learning, pp. 6–13 (2004)

    Google Scholar 

  8. Sonnenburg, S., Rätsch, G., Schäfer, C., Schölkopf, B.: Large scale multiple kernel learning. J. Mach. Learn. Res. 7, 1531–1565 (2006)

    MathSciNet  MATH  Google Scholar 

  9. Rakotomamonjy, A., Bach, F., Canu, S., Grandvalet, Y.: SimpleMKL. J. Mach. Learn. Res. 9, 2491–2521 (2008)

    MathSciNet  MATH  Google Scholar 

  10. Varma, M., Babu, B.R.: More generality in efficient multiple kernel learning. In: International Conference on Machine Learning (ICML), pp. 1065–1072 (2009)

    Google Scholar 

  11. Wu, P., Duan, F., Guo, P.: A pre-selecting base kernel method in multiple kernel learning. Neurocomputing 165, 46–53 (2015)

    Article  Google Scholar 

  12. Do, H., Kalousis, A., Woznica, A., Hilario, M.: Margin and radius based multiple kernel learning. In: Buntine, W., Grobelnik, M., Mladenić, D., Shawe-Taylor, J. (eds.) ECML PKDD 2009, Part I. LNCS, vol. 5781, pp. 330–343. Springer, Heidelberg (2009)

    Chapter  Google Scholar 

  13. Gai, K., Chen, G., Zhang, C.: Learning kernels with radiuses of minimum enclosing balls. In: Neural Information Processing Systems (NIPS), pp. 649–657 (2010)

    Google Scholar 

  14. Xu, Z., Weinberger, K., Chapelle, O.: Distance metric learning for kernel machines. arXiv preprint arXiv:1208.3422 (2012)

  15. Xu, Z., Jin, R., King, I., Lyu, M.: An extended level method for efficient multiple kernel learning. In: Neural Information Processing Systems (NIPS), pp. 1825–1832 (2009)

    Google Scholar 

  16. Wang, L.: Feature selection with kernel class separability. IEEE Trans. Pattern Anal. Mach. Intell. 30(9), 1534–1546 (2008)

    Article  Google Scholar 

  17. Liu, X., Wang, L., Yin, J., Zhu, E., Zhang, J.: An efficient approach to integrating radius information into multiple kernel learning. IEEE Trans. Cybern. 43(2), 557–569 (2013)

    Article  Google Scholar 

  18. Liu, X., Wang, L., Yin, J., Liu, L.: Incorporation of radius-info can be simple with SimpleMKL. Neurocomputing 89, 30–38 (2012)

    Article  Google Scholar 

  19. Do, H., Kalousis, A.: Convex formulation of radius-margin based Support Vector Machines. In: International Conference on Machine Learning (ICML), pp. 169–177 (2013)

    Google Scholar 

  20. Lu, X., Wang, Y., Zhou, X., Ling, Z.: A method for metric learning with multiple-kernel embedding. Neural Process. Lett. 43, 905–921 (2016)

    Article  Google Scholar 

  21. Wang, F., Zuo, W., Zhang, L., Meng, D., Zhang, D.: A kernel classification framework for metric learning. IEEE Trans. Neural Netw. Learn. Syst. 26(9), 1950–1962 (2014)

    Google Scholar 

  22. Shawe-Taylor, J., Cristianini, N.: Kernel Methods for Pattern Analysis. Cambridge University Press, Cambridge (2004)

    Google Scholar 

  23. Vapnik, V., Chapelle, O.: Bounds on error expectation for support vector machines. Neural Comput. 12(9), 2013–2036 (2000)

    Article  Google Scholar 

  24. Frank, A., Asuncion, A.: UCI machine learning repository (2010). http://archive.ics.uci.edu/ml

  25. Weinberger, K.Q., Blitzer, J., Saul, L.K.: Distance metric learning for large margin nearest neighbor classification. In: Neural Information Processing Systems (NIPS), pp. 1473–1480 (2005)

    Google Scholar 

  26. Davis, J.V., Kulis, B., Jain, P., Sra, S., Dhillon, I.S.: Information-theoretic metric learning. In: International Conference on Machine Learning (ICML), pp. 189–198 (2007)

    Google Scholar 

  27. Dem\(\tilde{{\rm s}}\)ar, J.: Statistical comparisons of classifiers over multiple data sets. J. Mach. Learn. Res. 7, 1–30 (2006)

    Google Scholar 

Download references

Acknowledgment

This work is partly support by the National Science Foundation of China (NSFC) project under the contract No. 61102037, 61271093, and 61471146.

Author information

Authors and Affiliations

  1. Harbin Institute of Technology, Harbin, 150001, China

    Weiqi Zhang, Zifei Yan, Hongzhi Zhang & Wangmeng Zuo

Authors
  1. Weiqi Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zifei Yan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hongzhi Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Wangmeng Zuo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zifei Yan .

Editor information

Editors and Affiliations

  1. Institute of Automation, Chinese Academy of Sciences, Beijing, China

    Tieniu Tan

  2. Xi’an Institute of Optics and Precision Mechanics, Chinese Academy of Sciences, Xi'an, China

    Xuelong Li

  3. Chinese Academy of Sciences, Institute of Computing Technology, Beijing, China

    Xilin Chen

  4. Tsinghua University , Beijing, China

    Jie Zhou

  5. Nanjing University of Science and Technology, Nanjing, China

    Jian Yang

  6. University of Electronic Science and Technology, Chengdu, Sichuan, China

    Hong Cheng

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer Nature Singapore Pte Ltd.

About this paper

Cite this paper

Zhang, W., Yan, Z., Zhang, H., Zuo, W. (2016). Joint Learning of Distance Metric and Kernel Classifier via Multiple Kernel Learning. In: Tan, T., Li, X., Chen, X., Zhou, J., Yang, J., Cheng, H. (eds) Pattern Recognition. CCPR 2016. Communications in Computer and Information Science, vol 662. Springer, Singapore. https://doi.org/10.1007/978-981-10-3002-4_48

Download citation

  • DOI: https://doi.org/10.1007/978-981-10-3002-4_48

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-10-3001-7

  • Online ISBN: 978-981-10-3002-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation