The increasing number of classification applications in large data sets demands that efficient classifiers be designed not only in training but also for prediction. In this paper, we address the problem of learning kernel classifiers with reduced complexity and improved efficiency for prediction in comparison to those trained by standard methods. A single optimisation problem is formulated for classifier learning which optimises both classifier weights and eXpansion Vectors (XVs) that define the classification function in a joint fashion. Unlike the existing approach of Wu et al, which performs optimisation in the dual formulation, our approach solves the primal problem directly. The primal problem is much more efficient to solve, as it can be converted to the training of a linear classifier in each iteration, which scales linearly to the size of the data set and the number of expansions. This makes our primal approach highly desirable for large-scale applications, where the dual approach is inadequate and prohibitively slow due to the solution of cubic-time kernel SVM involved in each iteration. Experimental results have demonstrated the efficiency and effectiveness of the proposed primal approach for learning sparse kernel classifiers that clearly outperform the alternatives.


  1. 1.
    Wu, M., Scholkopf, B., Bakir, G.: A direct method for building sparse kernel learning algorithms. Journal of Machine Learning Research 7, 603–624 (2006)MathSciNetzbMATHGoogle Scholar
  2. 2.
    Platt, J.: Fast training of support vector machines using sequential minimal optimization. In: Advances in Kernel Methods - Support Vector Learning (1998)Google Scholar
  3. 3.
    Lee, Y.J., Mangasarian, O.L.: Rsvm: Reduced support vector machines. In: Siam Data Mining Conf. (2001)Google Scholar
  4. 4.
    Scholkopf, B., Smola, A.: Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond. MIT Press (2002)Google Scholar
  5. 5.
    Keerthi, S., Chapelle, O., DeCoste, D.: Building support vector machines with reduced classifier complexity. Journal Machine Learning Res. 7, 1493–1515 (2006)MathSciNetzbMATHGoogle Scholar
  6. 6.
    Bonnans, J.F., Shapiro, A.: Optimization problems with pertubation: A guided tour. SIAM Review 40(2), 202–227 (1998)MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Zhouyu Fu
    • 1
  • Guojun Lu
    • 2
  • Kai-Ming Ting
    • 2
  • Dengsheng Zhang
    • 2
  1. 1.School of ComputingUniversity of Western SydneyPenrithAustralia
  2. 2.Gippsland School of ITMonash UniversityChurchillAustralia

Personalised recommendations