Over-Complete Wavelet Approximation of a Support Vector Machine for Efficient Classification

  • Matthias Rätsch
  • Sami Romdhani
  • Gerd Teschke
  • Thomas Vetter
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3663)


In this paper, we present a novel algorithm for reducing the runtime computational complexity of a Support Vector Machine classifier. This is achieved by approximating the Support Vector Machine decision function by an over-complete Haar wavelet transformation. This provides a set of classifiers of increasing complexity that can be used in a cascaded fashion yielding excellent runtime performance. This over-complete transformation finds the optimal approximation of the Support Vectors by a set of rectangles with constant gray-level values (enabling an Integral Image based evaluation). A major feature of our training algorithm is that it is fast, simple and does not require complicated tuning by an expert in contrast to the Viola & Jones classifier. The paradigm of our method is that, instead of trying to estimate a classifier that is jointly accurate and fast (such as the Viola & Jones detector), we first build a classifier that is proven to have optimal generalization capabilities; the focus then becomes runtime efficiency while maintaining the classifier’s optimal accuracy. We apply our algorithm to the problem of face detection in images but it can also be used for other image based classifications. We show that our algorithm provides, for a comparable accuracy, a 15 fold speed-up over the Reduced Support Vector Machine and a 530 fold speed-up over the Support Vector Machine, enabling face detection at 25 fps on a standard PC.


Support Vector Machine Wavelet Transformation Face Detection Image Patch Wavelet Basis 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Zikos, G., Garcia, C., Tziritas, G.: Face detection in color images using wavelet packet analysis. In: IEEE Int. Conf. on Multimedia Computing and Systems (1999)Google Scholar
  2. 2.
    Cohen, A., DeVore, R., Petrushev, P., Xu, H.: Nonlinear Approximation and the Space BV(ℝ2). American Journal of Mathematics (121), 587–628 (1999)Google Scholar
  3. 3.
    Coifman, R.R., Donoho, D.: Translation–invariant de–noising. In: Antoniadis, A., Oppenheim, G. (eds.) Wavelets and Statistics, pp. 125–150. Springer, New York (1995)Google Scholar
  4. 4.
    Crow, F.: Summed-area tables for texture mapping. Proc. of SIGGRAPH 18(3), 207–212 (1984)CrossRefGoogle Scholar
  5. 5.
    Daubechies, I., Teschke, G.: Variational image restoration by means of wavelets: simultaneous decomposition, deblurring and denoising. In: Applied and Computational Harmonic Analysis (2005)Google Scholar
  6. 6.
    Karras, D.A.: Improved defect detection in textile visual inspection using wavelet analysis and support vector machines. International Journal on Graphics, Vision and Image Processing (2005)Google Scholar
  7. 7.
    Keren, D., Osadchy, M., Gotsman, C.: Antifaces: a novel, fast method for image detection. IEEE Transactions on Pattern Analysis and Machine Intelligence 23, 747–761 (2001)CrossRefGoogle Scholar
  8. 8.
    Kienzle, W., Bakir, G.H., Franz, M.O., Schölkopf, B.: Efficient approximations for support vector machines in object detection. In: Rasmussen, C.E., Bülthoff, H.H., Schölkopf, B., Giese, M.A. (eds.) DAGM 2004. LNCS, vol. 3175, pp. 54–61. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  9. 9.
    Rätsch, M., Romdhani, S., Vetter, T.: Efficient face detection by a cascaded support vector machine using haar-like features. In: Rasmussen, C.E., Bülthoff, H.H., Schölkopf, B., Giese, M.A. (eds.) DAGM 2004. LNCS, vol. 3175, pp. 62–70. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  10. 10.
    Romdhani, S., Torr, P., Schölkopf, B., Blake, A.: Computationally efficient face detection. In: Proceedings of the 8th International Conference on Computer Vision (July 2001)Google Scholar
  11. 11.
    Rowley, H., Baluja, S., Kanade, T.: Neural network-based face detection. PAMI 20, 23–38 (1998)Google Scholar
  12. 12.
    Schmeisser, H.-J., Triebel, H.: Topics in Fourier Analysis and Function Spaces. John Wiley and Sons, New York (1987)Google Scholar
  13. 13.
    Schölkopf, B., Mika, S., Burges, C., Knirsch, P., Müller, K.-R., Rätsch, G., Smola, A.: Input space vs. feature space in kernel-based methods. IEEE TNN 10(5), 1000–1017 (1999)Google Scholar
  14. 14.
    Triebel, H.: Interpolation Theory, Function Spaces, Differential Operators. Verlag der Wissenschaften, Berlin (1978)Google Scholar
  15. 15.
    Vapnik, V.: Statistical Learning Theory. Wiley, New York (1998)zbMATHGoogle Scholar
  16. 16.
    Viola, P., Jones, M.: Rapid object detection using a boosted cascade of simple features. In: Proceedings IEEE Conf. on Computer Vision and Pattern Recognition (2001)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2005

Authors and Affiliations

  • Matthias Rätsch
    • 1
  • Sami Romdhani
    • 1
  • Gerd Teschke
    • 2
  • Thomas Vetter
    • 1
  1. 1.Computer Science DepartmentUniversity of BaselBaselSwitzerland
  2. 2.University of Bremen, ZETEMBremenGermany

Personalised recommendations