Efficient Mixed-Norm Multiple Kernel Learning
Multiple kernels learning (MKL) is a hot topic in the current kernel machine learning field which aims at find a convexity linear combination of based kernels. Current MKL methods encourage spare kernel coefficients combination, unfortunately, when features encode orthogonal data, spareness tends to select only a few kernels, and may discards useful information which lead to poor generalization performance. In this paper, we presented an efficient multiple kernels learning method based on mix-norm in which sparseness and nonsparseness can be compromised using a mixing regularization. Both SVM and MKL could be regarded as special cases of EMNMKL. Then, we developed a rapid gradient descent algorithm to deal with the problem. Simulation experiment results show that the EMNMKL rapidly converges and the average testing accuracy demonstrates that EMNMKL algorithm clearly outperforms SVM and MKL.
KeywordsMultiple kernel learning Mix-norm Mixing regularization Gradient descent algorithm
The work was supposed by the Scientific Research Foundation of the Chongqing Municipal Education Commission, China (No.KJ090823&No.KJ110632), Natureal Science Foundation of Chongqing, China (No.cstc2011jjA40008), and the Foundation of the Chongqing Normal University, China (No.11XLB047).
- 2.G¨onen M, Alpaydın E (2011) Multiple kernel learning algorithms. J Mach Learn Res 12:2211–2268Google Scholar
- 3.Rakotomamonjy A, Bach FR, Canu S, Grandvalet Y (2008) SimpleMKL. J Mach Learn Res 9:2491–2521Google Scholar
- 5.Zhang X, Hu L, Wang ZS (2010) Multiple kernel support vector regression for economic forecasting, vol 36. In: International conference on management science and engineering, 17st. Melbourne, Australia, pp 129–134Google Scholar
- 8.Rakotomamonjy A, Bach F, Canu S, Grandvalet Y (2007) More efficiency in multiple kernel learning, vol 36. In: International conference on machine learning, 24st. Corvallis, OR, pp 73–75Google Scholar
- 9.Xu Z, Jin R, Yang H, King I, Lyu MR (2010) Simple and efficient multiple kernel learning by Group Lasso, vol 24. In: International conference on machine learning, 27st. Haifa, Israel, pp 62–66Google Scholar
- 10.Kloft M, Sonnenburg S, Laskov P, M¨uller KR, Zien A (2009) Efficient and accurate lp-norm multiple kernel learning. Adv Neural Inf Process Syst 22:997–1005Google Scholar
- 11.Kloft M, Brefeld U, Laskov P, Sonnenburg S (2008) Non-sparse multiple kernel learning, vol 325. In: NIPS workshop on kernel learning: automatic selection of optimal kernels. Whistler, Canada, pp 62–66Google Scholar
- 12.Saketha Nath J, Dinesh G, Raman S, Bhattacharyya C, Ben-Tal A, Ramakrishnan KR (2009) On the algorithmic and applications of a mixed-norm based kernel learning formulation. http://books.nips.cc/papers/files/nips22/NIPS2009_0603 73:245-251
- 13.Wu ZP, Zhang XG (2010) Elastic multiple kernel learning. Acta Automatica Sinica 37(6):693–699Google Scholar
- 14.Kloft M, Brefeld U, Sonnenburg S, Zien A (2011) lp-norm multiple kernel learning. J Mach Learn Res 12:663–709Google Scholar