Efficient Mixed-Norm Multiple Kernel Learning

Conference paper
Part of the Lecture Notes in Electrical Engineering book series (LNEE, volume 216)


Multiple kernels learning (MKL) is a hot topic in the current kernel machine learning field which aims at find a convexity linear combination of based kernels. Current MKL methods encourage spare kernel coefficients combination, unfortunately, when features encode orthogonal data, spareness tends to select only a few kernels, and may discards useful information which lead to poor generalization performance. In this paper, we presented an efficient multiple kernels learning method based on mix-norm in which sparseness and nonsparseness can be compromised using a mixing regularization. Both SVM and MKL could be regarded as special cases of EMNMKL. Then, we developed a rapid gradient descent algorithm to deal with the problem. Simulation experiment results show that the EMNMKL rapidly converges and the average testing accuracy demonstrates that EMNMKL algorithm clearly outperforms SVM and MKL.


Multiple kernel learning Mix-norm Mixing regularization Gradient descent algorithm 



The work was supposed by the Scientific Research Foundation of the Chongqing Municipal Education Commission, China (No.KJ090823&No.KJ110632), Natureal Science Foundation of Chongqing, China (No.cstc2011jjA40008), and the Foundation of the Chongqing Normal University, China (No.11XLB047).


  1. 1.
    Wang H-Q, Sun F-C, Cai Y-N, Chen N, Ding L-G (2010) On multiple kernel learning methods. Acta Automatica Sinica 36(8):1037–1050MathSciNetCrossRefGoogle Scholar
  2. 2.
    G¨onen M, Alpaydın E (2011) Multiple kernel learning algorithms. J Mach Learn Res 12:2211–2268Google Scholar
  3. 3.
    Rakotomamonjy A, Bach FR, Canu S, Grandvalet Y (2008) SimpleMKL. J Mach Learn Res 9:2491–2521Google Scholar
  4. 4.
    Tuia D, Camps-Valls G, Matasci G, Kanevski M (2010) Learning relevant image features with multiple-kernel classification. IEEE Geosci Remote Sens Soc 48(10):3780–3791CrossRefGoogle Scholar
  5. 5.
    Zhang X, Hu L, Wang ZS (2010) Multiple kernel support vector regression for economic forecasting, vol 36. In: International conference on management science and engineering, 17st. Melbourne, Australia, pp 129–134Google Scholar
  6. 6.
    Yeh CY, Huang CW, Lee SJ (2011) A multiple-kernel support vector regression approach for stock market price forecasting. Expert Syst Appl 38:2177–2186CrossRefGoogle Scholar
  7. 7.
    Wang HQ, Cai YN, Sun FC, Zhao ZT (2011) Adaptive sequence learning and applications for multi-scale kernel method. Pattern Recogn Artif Intell 24(1):72–81CrossRefGoogle Scholar
  8. 8.
    Rakotomamonjy A, Bach F, Canu S, Grandvalet Y (2007) More efficiency in multiple kernel learning, vol 36. In: International conference on machine learning, 24st. Corvallis, OR, pp 73–75Google Scholar
  9. 9.
    Xu Z, Jin R, Yang H, King I, Lyu MR (2010) Simple and efficient multiple kernel learning by Group Lasso, vol 24. In: International conference on machine learning, 27st. Haifa, Israel, pp 62–66Google Scholar
  10. 10.
    Kloft M, Sonnenburg S, Laskov P, M¨uller KR, Zien A (2009) Efficient and accurate lp-norm multiple kernel learning. Adv Neural Inf Process Syst 22:997–1005Google Scholar
  11. 11.
    Kloft M, Brefeld U, Laskov P, Sonnenburg S (2008) Non-sparse multiple kernel learning, vol 325. In: NIPS workshop on kernel learning: automatic selection of optimal kernels. Whistler, Canada, pp 62–66Google Scholar
  12. 12.
    Saketha Nath J, Dinesh G, Raman S, Bhattacharyya C, Ben-Tal A, Ramakrishnan KR (2009) On the algorithmic and applications of a mixed-norm based kernel learning formulation. 73:245-251
  13. 13.
    Wu ZP, Zhang XG (2010) Elastic multiple kernel learning. Acta Automatica Sinica 37(6):693–699Google Scholar
  14. 14.
    Kloft M, Brefeld U, Sonnenburg S, Zien A (2011) lp-norm multiple kernel learning. J Mach Learn Res 12:663–709Google Scholar

Copyright information

© Springer-Verlag London 2013

Authors and Affiliations

  1. 1.College of Computer and Information ScienceChongqing Normal UniversityChongqingChina

Personalised recommendations