Linearized Smooth Additive Classifiers

  • Subhransu Maji
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7583)


We consider a framework for learning additive classifiers based on regularized empirical risk minimization, where the regularization favors “smooth” functions. We present representations of classifiers for which the optimization problem can be efficiently solved. The first family of such classifiers are derived from a penalized spline formulation due to Eilers and Marx, which is modified to enabled linearization. The second is a novel family of classifiers that are based on classes of orthogonal basis functions with othogonal derivatives. Both these families lead to explicit feature embeddings that can be used with off-the-shelf linear solvers such as LIBLINEAR to obtain additive classifiers. The proposed family of classifiers offer better trade-offs between training time, memory overhead and classifier accuracy, compared to the state-of-the-art in additive classifier training.


Training Time Computer Vision Application Memory Overhead Empirical Risk Minimization Orthogonal Basis Function 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Eilers, P., Marx, B.: Generalized linear additive smooth structures. Journal of Computational and Graphical Statistics 11(4), 758–783 (2002)MathSciNetCrossRefGoogle Scholar
  2. 2.
    Lazebnik, S., Schmid, C., Ponce, J.: Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories. In: CVPR (2006)Google Scholar
  3. 3.
    Maji, S., Berg, A.C.: Max margin additive classifiers for detection. In: ICCV (2009)Google Scholar
  4. 4.
    Vedaldi, A., Zisserman, A.: Efficient additive kernels via explicit feature maps. In: CVPR (2010)Google Scholar
  5. 5.
    Rahimi, A., Recht, B.: Random features for large-scale kernel machines. In: NIPS (2007)Google Scholar
  6. 6.
    Hein, M., Bousquet, O.: Hilbertian metrics and positive definite kernels on probability measures. In: AISTATS (2005)Google Scholar
  7. 7.
    Hastie, T., Tibshirani, R.: Generalized Additive Models. Chapman & Hall/CRC (1990)Google Scholar
  8. 8.
    Pearce, N., Wand, M.: Penalized splines and reproducing kernel methods. The American Statistician 60(3), 233–240 (2006)MathSciNetCrossRefGoogle Scholar
  9. 9.
    Wahba, G.: Spline models for observational data, vol. 59. Society for Industrial Mathematics (1990)Google Scholar
  10. 10.
    Eilers, P., Marx, B.: Splines, knots, and penalties. Wiley Interdisciplinary Reviews: Computational Statistics (2005)Google Scholar
  11. 11.
    Webster, M.: Orthogonal polynomials with orthogonal derivatives. Mathematische Zeitschrift 39, 634–638 (1935)MathSciNetCrossRefGoogle Scholar
  12. 12.
    Shalev-Shwartz, S., Singer, Y., Srebro, N.: Pegasos: Primal estimated sub-gradient solver for svm. In: ICML (2007)Google Scholar
  13. 13.
    LeCun, Y., Cortes, C.: The mnist database of handwritten digits (1998)Google Scholar
  14. 14.
    Munder, S., Gavrila, D.M.: An experimental study on pedestrian classification. IEEE TPAMI 28(11) (2006)Google Scholar
  15. 15.
    Maji, S., Berg, A.C., Malik, J.: Classification using intersection kernel support vector machines is efficient. In: CVPR (2008)Google Scholar
  16. 16.
    Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: CVPR (2005)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Subhransu Maji
    • 1
  1. 1.Toyota Technological Institute at ChicagoChicagoUSA

Personalised recommendations