Invariance in Kernel Methods by Haar-Integration Kernels

  • B. Haasdonk
  • A. Vossen
  • H. Burkhardt
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3540)


We address the problem of incorporating transformation invariance in kernels for pattern analysis with kernel methods. We introduce a new class of kernels by so called Haar-integration over transformations. This results in kernel functions, which are positive definite, have adjustable invariance, can capture simultaneously various continuous or discrete transformations and are applicable in various kernel methods. We demonstrate these properties on toy examples and experimentally investigate the real-world applicability on an image recognition task with support vector machines. For certain transformations remarkable complexity reduction is demonstrated. The kernels hereby achieve state-of-the-art results, while omitting drawbacks of existing methods.


  1. 1.
    Schölkopf, B., Smola, A.J.: Learning with Kernels. MIT Press, Cambridge (2002)Google Scholar
  2. 2.
    DeCoste, D., Schölkopf, B.: Training invariant support vector machines. Machine Learning 46, 161–190 (2002)zbMATHCrossRefGoogle Scholar
  3. 3.
    Schulz-Mirbach, H.: Constructing invariant features by averaging techniques. In: Proc. of the 12th ICPR, vol. 2, pp. 387–390. IEEE Computer Society, Los Alamitos (1994)Google Scholar
  4. 4.
    Schael, M.: Texture defect detection using invariant textural features. In: Proc. of the 23rd DAGM - Symposium Mustererkennung, pp. 17–24. Springer, Heidelberg (2001)Google Scholar
  5. 5.
    Siggelkow, S.: Feature-Histograms for Content-Based Image Retrieval. PhD thesis, Albert-Ludwigs-Universität Freiburg (2002)Google Scholar
  6. 6.
    Ronneberger, O., Schultz, E., Burkhardt, H.: Automated pollen recognition using 3d volume images from fluorescence microscopy. Aerobiologia 18, 107–115 (2002)CrossRefGoogle Scholar
  7. 7.
    Haasdonk, B., Halawani, A., Burkhardt, H.: Adjustable invariant features by partial Haar-integration. In: Proc. of the 17th ICPR, vol. 2, pp. 769–774 (2004)Google Scholar
  8. 8.
    Haasdonk, B., Keysers, D.: Tangent distance kernels for support vector machines. In: Proc. of the 16th ICPR, vol. 2, pp. 864–868 (2002)Google Scholar
  9. 9.
    Pozdnoukhov, A., Bengio, S.: Tangent vector kernels for invariant image classification with SVMs. In: Proc. of the 17th ICPR (2004)Google Scholar
  10. 10.
    Burges, C.J.C.: Geometry and invariance in kernel based methods. In: Advances in Kernel Methods - Support Vector Learning, pp. 89–116. MIT Press, Cambridge (1999)Google Scholar
  11. 11.
    Chapelle, O., Schölkopf, B.: Incorporating invariances in nonlinear support vector machines. In: NIPS 14, pp. 609–616. MIT-Press, Cambridge (2002)Google Scholar
  12. 12.
    Schölkopf, B., Simard, P., Smola, A., Vapnik, V.: Prior knowledge in support vector kernels. In: NIPS 10, pp. 640–646. MIT Press, Cambridge (1998)Google Scholar
  13. 13.
    Schölkopf, B., Burges, C., Vapnik, V.: Incorporating invariances in support vector learning machines. In: Vorbrüggen, J.C., von Seelen, W., Sendhoff, B. (eds.) ICANN 1996. LNCS, vol. 1112, pp. 47–52. Springer, Heidelberg (1996)Google Scholar
  14. 14.
    Simard, P., LeCun, Y., Denker, J.: Efficient pattern recognition using a new transformation distance. In: NIPS 5, pp. 50–58. Morgan Kaufmann, San Francisco (1993)Google Scholar
  15. 15.
    Schölkopf, B., Burges, C., Vapnik, V.: Extracting support data for a given task. In: Proc. of the 1st KDD, pp. 252–257. AAAI Press, Menlo Park (1995)Google Scholar
  16. 16.
    Ronneberger, O., Pigorsch, F.: LIBSVMTL: a support vector machine template library (2004),

Copyright information

© Springer-Verlag Berlin Heidelberg 2005

Authors and Affiliations

  • B. Haasdonk
    • 1
  • A. Vossen
    • 2
  • H. Burkhardt
    • 1
  1. 1.Computer Science DepartmentAlbert-Ludwigs-University FreiburgFreiburgGermany
  2. 2.Institute of PhysicsAlbert-Ludwigs-University FreiburgFreiburgGermany

Personalised recommendations