Support Vector Regression Using Mahalanobis Kernels

  • Yuya Kamada
  • Shigeo Abe
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4087)


In our previous work we have shown that Mahalanobis kernels are useful for support vector classifiers both from generalization ability and model selection speed. In this paper we propose using Mahalanobis kernels for function approximation. We determine the covariance matrix for the Mahalanobis kernel using all the training data. Model selection is done by line search. Namely, first the margin parameter and the error threshold are optimized and then the kernel parameter is optimized. According to the computer experiments for four benchmark problems, estimation performance of a Mahalanobis kernel with a diagonal covariance matrix optimized by line search is comparable to or better than that of an RBF kernel optimized by grid search.


Model Selection Radial Basis Function Support Vector Regression Line Search Mahalanobis Distance 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Duan, K., Keerthi, S.S., Poo, A.N.: An Empirical Evaluation of Simple Performance Measures for Tuning SVM Hyperparameters. In: Proc. ICONIP 2001, Paper ID#159 (2001)Google Scholar
  2. 2.
    Herbrich, R.: Learning Kernel Classifiers: Theory and Algorithms. MIT Press, Cambridge (2002)Google Scholar
  3. 3.
    Friedrichs, F., Igel, C.: Evolutionary Tuning of Multiple SVM Parameters. In: Proc. ESANN 2004, pp. 519–524 (2004)Google Scholar
  4. 4.
    Abe, S.: Training of Support Vector Machines with Mahalanobis Kernels. In: Duch, W., Kacprzyk, J., Oja, E., Zadrożny, S. (eds.) ICANN 2005. LNCS, vol. 3697, pp. 571–576. Springer, Heidelberg (2005)Google Scholar
  5. 5.
    Abe, S.: Pattern Classification: Neuro-Fuzzy Methods and Their Comparison. Springer, London (2001)MATHGoogle Scholar
  6. 6.
    Abe, S.: Support Vector Machines for Pattern Classification. Springer, London (2005)Google Scholar
  7. 7.
    Baba, K., Enbutu, I., Yoda, M.: Explicit Representation of Knowledge Acquired from Plant Historical Data Using Neural Network. In: Proc. IJCNN 1990, vol. 3, pp. 155–160 (1990)Google Scholar
  8. 8.
    Abe, S.: Neural Networks and Fuzzy Systems: Theory and Applications. Kluwer, Boston (1997)MATHGoogle Scholar
  9. 9.
    Crowder, R.S.: Predicting the Mackey-Glass Time Series with Cascade-Correlation Learning. In: Proc. 1990 Connectionist Models Summer School, pp. 117–123. Carnegie Mellon University (1990)Google Scholar
  10. 10.
    Harrison, D., Rubinfeld, D.L.: Hedonic Prices and the Demand for Clean Air. Journal of Environmental Economics and Management 5, 81–102 (1978)MATHCrossRefGoogle Scholar
  11. 11.

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Yuya Kamada
    • 1
  • Shigeo Abe
    • 1
  1. 1.Graduate School of Science and TechnologyKobe UniversityKobeJapan

Personalised recommendations