Annals of Mathematics and Artificial Intelligence

, Volume 69, Issue 3, pp 225–241 | Cite as

Analytic center cutting plane method for multiple kernel learning

Article
  • 147 Downloads

Abstract

Multiple Kernel Learning (MKL) is a popular generalization of kernel methods which allows the practitioner to optimize over convex combinations of kernels. We observe that many recent MKL solutions can be cast in the framework of oracle based optimization, and show that they vary in terms of query point generation. The popularity of such methods is because the oracle can fortuitously be implemented as a support vector machine. Motivated by the success of centering approaches in interior point methods, we propose a new approach to optimize the MKL objective based on the analytic center cutting plane method (accpm). Our experimental results show that accpm outperforms state of the art in terms of rate of convergence and robustness. Further analysis sheds some light as to why MKL may not always improve classification accuracy over naive solutions.

Keywords

Multiple kernel learning accpm Oracle methods Machine learning 

Mathematics Subject Classification (2010)

68T05 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Schölkopf, B., Smola, A.J.: Learning with Kernels. MIT Press, Cambridge, MA (2002)Google Scholar
  2. 2.
    Lanckriet, G., De Bie, T., Cristianini, N., Jordan, M.I., Stafford Noble, W.: A statistical framework for genomic data fusion. Bioinfomatics 20(16), 2626–2635 (2004).CrossRefGoogle Scholar
  3. 3.
    Borgwardt, K.M., Ong, C.S., Schönauer, S., Vishwanathan, S.V.N., Smola, A.J., Kriegel, H.-P.: Protein function prediction via graph kernels. In: Proceedings of the International Conference on Intelligent Systems for Molecular Biology (2005)Google Scholar
  4. 4.
    Sonnenburg, S., Rätsch, G., Schäfer, C.: A general and efficient multiple kernel learning algorithm. In: Neural Information Processings Systems (2005)Google Scholar
  5. 5.
    Ong, C.S., Zien, A.: An automated combination of kernels for predicting protein subcellular localization. In: Crandall, K.A., Lagergren, J. (eds.) WABI, pp. 86–197. Springer (2008)Google Scholar
  6. 6.
    Lanckriet, G., Cristianini, N., Bartlett, P., El Ghaoui, L., Jordan, M.I.: Learning the kernel matrix with semi-definite programming. J. Mach. Learn. Res. 5, 27–72 (2004).MATHGoogle Scholar
  7. 7.
    Sonnenburg, S., Rätsch, G., Schäfer, C., Schölkopf, B.: Large scale multiple kernel learning. J. Mach. Learn. Res. 7, 1531–1565 (2006)MATHMathSciNetGoogle Scholar
  8. 8.
    Bach, F.R., Lanckriet, G.R.G., Jordan, M.I.: Multiple kernel learning, conic duality, and the SMO algorithm. In: ICML (2004)Google Scholar
  9. 9.
    Bach, F.R.: Consistency of the group lasso and multiple kernel learning. J. Mach. Learn. Res. 9, 1179–1225 (2008)MATHMathSciNetGoogle Scholar
  10. 10.
    Rakotomamonjy, A., Bach, F.R., Canu, S., Grandvalet, Y.: Simplemkl. J. Mach. Learn. Res. 9, 2491–2521 (2008)MATHMathSciNetGoogle Scholar
  11. 11.
    Xu, Z., Jin, R., King, I., Lyu, M.R.: An extended level method for efficient multiple kernel learning. In: NIPS (2008)Google Scholar
  12. 12.
    Vial, J., Goffin, J.: Convex nondifferentiable optimization: a survey focused on the analytic center cutting plane method. Optim. Methods Softw. 17(5), 805–867 (2002)CrossRefMATHMathSciNetGoogle Scholar
  13. 13.
    Zien, A., Ong, C.S.: Multiclass multiple kernel learning. In: ICML (2007)Google Scholar
  14. 14.
    Babonneau, F., Beltran, C., Haurie, A., Tadonki, C., Vial, J.-P.: Proximal-ACCPM: a versatile oracle based optimization method. In: Kontoghiorghes, E.J., Gatu, C. (eds.) Optimisation, Econometric and Financial Analysis (2007)Google Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2013

Authors and Affiliations

  1. 1.Department of Computer ScienceETHZürichSwitzerland
  2. 2.NICTAThe University of MelbourneMelbourneAustralia

Personalised recommendations