Applied Intelligence

, Volume 41, Issue 1, pp 299–309 | Cite as

K-nearest neighbor-based weighted twin support vector regression

  • Yitian XuEmail author
  • Laisheng Wang


Twin support vector regression (TSVR) finds ϵ-insensitive up- and down-bound functions by resolving a pair of smaller-sized quadratic programming problems (QPPs) rather than a single large one as in a classical SVR, which makes its computational speed greatly improved. However the local information among samples are not exploited in TSVR. To make full use of the knowledge of samples and improve the prediction accuracy, a K-nearest neighbor-based weighted TSVR (KNNWTSVR) is proposed in this paper, where the local information among samples are utilized. Specifically, a major weight is given to the training sample if it has more K-nearest neighbors. Otherwise a minor weight is given to it. Moreover, to further enhance the computational speed, successive overrelaxation approach is employed to resolve the QPPs. Experimental results on eight benchmark datasets and a real dataset demonstrate our weighted TSVR not only yields lower prediction error but also costs lower running time in comparison with other algorithms.


TSVR K-nearest neighbor Weights Successive overrelaxation 



The authors gratefully acknowledge the helpful comments and suggestions of the reviewers, which have improved the presentation. This work was supported by the National Natural Science Foundation of China (No. 61153003, 11171346 and 11271367) and China Scholarship Fund (No. 201208110282).


  1. 1.
    Cai D, He X, Zhang W, Han J (2007) Regularized locality preserving indexing via spectral regression. In: Proc 2007 ACM int. conf. on information and knowledge management Google Scholar
  2. 2.
    Cheng H, Tan P, Jin R (2010) Efficient algorithm for localized support vector machine. IEEE Trans Knowl Data Eng 22(4):537–549 CrossRefGoogle Scholar
  3. 3.
    Cover T, Hart P (1967) Nearest neighbor pattern classification. IEEE Trans Inf Theory 13:21–27 CrossRefzbMATHGoogle Scholar
  4. 4.
    Diosan L, Rogozan A, Pecuchet J-P (2013) Improving classification performance of support vector machine by genetically optimising kernel shape and hyper-parameters. Appl Intell 36:280–294 CrossRefGoogle Scholar
  5. 5.
    Fung G, Mangasarian O (2001) Proximal support vector machine classifiers. In: Seven international proceedings on knowledge discovery and data mining, pp 77–86 Google Scholar
  6. 6.
    Fung G, Mangasarian O (2005) Multicategory proximal support vector machine classifiers. Mach Learn 59:77–97 CrossRefzbMATHGoogle Scholar
  7. 7.
    Ghorai S, Mukherjee A, Dutta P (2009) Nonparallel plane proximal classifier. Signal Process 89(4):510–522 CrossRefzbMATHGoogle Scholar
  8. 8.
    Jayadeva, Khemchandani R, Chandra S (2007) Twin support vector machines for pattern classification. IEEE Trans Pattern Anal Mach Intell 29(5):905–910 CrossRefGoogle Scholar
  9. 9.
    Jayadeva, Khemchandani R, Chandra S (2007) Fuzzy multi-category proximal support vector classification via generalized eigenvalues. Soft Comput 11(7):679–685 CrossRefGoogle Scholar
  10. 10.
    Khemchandani R, Jayadeva, Chandra S (2009) Optimal kernel selection in twin support vector machines. Optim Lett 3(1):77–88 CrossRefzbMATHMathSciNetGoogle Scholar
  11. 11.
    Kumar M, Gopal M (2009) Least squares twin support vector machines for pattern classification. Expert Syst Appl 36(4):7535–7543 CrossRefGoogle Scholar
  12. 12.
    Li I, Chen J, Wu J (2013) A fast prototype reduction method based on template reduction and visualization-induced self-organizing map for nearest neighbor algorithm. Appl Intell 39:564–582 CrossRefGoogle Scholar
  13. 13.
    Mangasarian O, Musicant D (1999) Successive overrelaxation for support vector machines. IEEE Trans Neural Netw 10:1032–1037 CrossRefGoogle Scholar
  14. 14.
    Matei O, Pop P, Vălean (2013) Optical character recognition in real environments using neural networks and k-nearest neighbor. Appl Intell 39:739–748 CrossRefGoogle Scholar
  15. 15.
    Owusu E, Zhan Y, Mao Q (2013) An SVM-AdaBoost facial expression recognition system. Appl Intell. doi: 10.1007/s10489-013-0478-9 zbMATHGoogle Scholar
  16. 16.
    Peng X (2010) TSVR: an efficient twin support vector machine for regression. Neural Netw 23(3):365–372 CrossRefGoogle Scholar
  17. 17.
    Peng X (2010) A new twin support vector machine classifier and its geometric algorithms. Inf Sci 180(20):3863–3875 CrossRefzbMATHGoogle Scholar
  18. 18.
    Ripley B (1996) Pattern recognition and neural networks. Cambridge University Press, Cambridge zbMATHGoogle Scholar
  19. 19.
    Shao Y, Wang Z, Chen W, Deng N (2013) Least squares twin parametric-margin support vector machine for classification. Appl Intell 39:451–464 CrossRefGoogle Scholar
  20. 20.
    Vapnik V (1995) The nature of statistical learning theory. Springer, New York CrossRefzbMATHGoogle Scholar
  21. 21.
    Wang C, You W (2013) Boosting-SVM: effective learning with reduced data dimension. Appl Intell 39:465–474 CrossRefGoogle Scholar
  22. 22.
    Xu Y (2012) A rough margin-based linear ν support vector regression. Stat Probab Lett 82:528–534 CrossRefzbMATHGoogle Scholar
  23. 23.
    Xu Y, Guo R (2014) An improved v-twin support vector machine. Applied Intell. doi: 10.1007/s10489-013-0500-2
  24. 24.
    Xu Y, Wang L (2005) Fault diagnosis system based on rough set theory and support vector machine. Lect Notes Artif Intell 3614:981–988 Google Scholar
  25. 25.
    Xu Y, Wang L (2012) A weighted twin support vector regression. Knowl-Based Syst 33:92–101 CrossRefGoogle Scholar
  26. 26.
    Xu Y, Wang L, Zhong P (2012) A rough margin-based ν-twin support vector machine. Neural Comput Appl 21:1307–1317 CrossRefGoogle Scholar
  27. 27.
    Xu Y, Lv X, Xi W (2012) A weighted multi-output support vector regression and its application. J Comput Inf Syst 8(9):3807–3814 Google Scholar
  28. 28.
    Ye Q, Zhao C, Gao S, Zheng H (2012) Weighted twin support vector machines with local information and its application. Neural Netw 35:31–39 CrossRefzbMATHGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2014

Authors and Affiliations

  1. 1.College of ScienceChina Agricultural UniversityBeijingChina
  2. 2.Department of Computer Science and EngineeringArizona State UniversityTempeUSA

Personalised recommendations