Advertisement

Reducing Hubness for Kernel Regression

  • Kazuo HaraEmail author
  • Ikumi Suzuki
  • Kei Kobayashi
  • Kenji Fukumizu
  • Miloš Radovanović
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9371)

Abstract

In this paper, we point out that hubness—some samples in a high-dimensional dataset emerge as hubs that are similar to many other samples—influences the performance of kernel regression. Because the dimension of feature spaces induced by kernels is usually very high, hubness occurs, giving rise to the problem of multicollinearity, which is known as a cause of instability of regression results. We propose hubness-reduced kernels for kernel regression as an extension of a previous approach for kNN classification that reduces spatial centrality to eliminate hubness.

Keywords

Mean Square Error Kernel Function Training Sample Gaussian Kernel Ridge Regression 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Chatterjee, S., Hadi, A.S., Price, B.: Regression Analysis By Example. Wiley Series In Probability And Statistics. Wiley, New York (2000)zbMATHGoogle Scholar
  2. 2.
    Gretton, A., Fukumizu, K., Teo, C., Song, L., Schölkopf, B., Smola, A.: A kernel statistical test of independence. Advances in Neural Information Processing Systems 20, 585–592 (2008)Google Scholar
  3. 3.
    Hara, K., Suzuki, I., Shimbo, M., Kobayashi, K., Fukumizu, K., Radovanović, M.: Localized centering: reducing hubness in large-sample data. In: AAAI (2015)Google Scholar
  4. 4.
    Montgomery, D.C., Peck, E.: Introduction to linear regression analysis. Wiley-Interscience Publication, John Wiley & sons, New York (1992)zbMATHGoogle Scholar
  5. 5.
    Radovanović, M., Nanopoulos, A., Ivanović, M.: Hubs in space: Popular nearest neighbors in high-dimensional data. Journal of Machine Learning Research 11, 2487–2531 (2010)MathSciNetzbMATHGoogle Scholar
  6. 6.
    Schnitzer, D., Flexer, A., Schedl, M., Widmer, G.: Local and global scaling reduce hubs in space. Journal of Machine Learning Research 13(1), 2871–2902 (2012)MathSciNetzbMATHGoogle Scholar
  7. 7.
    Suzuki, I., Hara, K., Shimbo, M., Matsumoto, Y., Saerens, M.: Investigating the effectiveness of laplacian-based kernels in hub reduction. In: AAAI (2012)Google Scholar
  8. 8.
    Suzuki, I., Hara, K., Shimbo, M., Saerens, M., Fukumizu, K.: Centering similarity measures to reduce hubs. In: EMNLP, pp. 613–623 (2013)Google Scholar
  9. 9.
    Tomasev, N., Radovanovic, M., Mladenic, D., Ivanovic, M.: The role of hubness in clustering high-dimensional data. IEEE Transactions on Knowledge and Data Engineering 26(3), 739–751 (2014)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Kazuo Hara
    • 1
    Email author
  • Ikumi Suzuki
    • 2
  • Kei Kobayashi
    • 2
  • Kenji Fukumizu
    • 2
  • Miloš Radovanović
    • 3
  1. 1.National Institute of GeneticsMishimaJapan
  2. 2.The Institute of Statistical MathematicsTachikawaJapan
  3. 3.University of Novi SadNovi SadSerbia

Personalised recommendations