Soft Computing

, Volume 14, Issue 9, pp 921–929 | Cite as

Quantum clustering-based weighted linear programming support vector regression for multivariable nonlinear problem

Original Paper

Abstract

Linear programming support vector regression shows improved reliability and generates sparse solution, compared with standard support vector regression. We present the v-linear programming support vector regression approach based on quantum clustering and weighted strategy to solve the multivariable nonlinear regression problem. First, the method applied quantum clustering to variable selection, introduced inertia weight, and took prediction precision of v-linear programming support vector regression as evaluation criteria, which effectively removed redundancy feature attributes and also reduced prediction error and support vectors. Second, it proposed a new weighted strategy due to each data point having different influence on regression model and determined the weighted parameter p in terms of distribution of training error, which greatly improved the generalization approximate ability. Experimental results demonstrated that the proposed algorithm enabled the mean squared error of test sets of Boston housing, Bodyfat, Santa dataset to, respectively, decrease by 23.18, 78.52, and 41.39%, and also made support vectors degrade rapidly, relative to the original v-linear programming support vector regression method. In contrast with other methods exhibited in the relevant literatures, the present algorithm achieved better generalization performance.

Keywords

Linear programming support vector regression Quantum clustering Variable selection Weighted strategy 

Notes

Acknowledgments

This paper was supported by National Science Foundation for Distinguished Young Scholars (No. 60625302), National Natural Science Foundation of China (General Program) (No. 60704028), Program for Changjiang Scholars and Innovative Research Team in University (No.IRT0721), the 111 Project (No.B08021), Shanghai Leading Academic Discipline Project (No.B504).

References

  1. Asuncion A, Newman DJ (2007) UCI machine learning repository. School of Information and Computer Sciences, University of California, Irvine. http://mlearn.ics.uci.edu/MLRepository.html. Accessed 6 Mar 2007
  2. Aïmeur E, Brassard G, Gambs S (2007) Quantum clustering algorithms. In: Proceedings of the 24th international conference on machine learning, vol 227. pp 1–8Google Scholar
  3. Butterworth R, Piatetsky-Shapiro G, Simovici DA (2005) On feature selection through clustering. Fifth IEEE Int Conf Data Min 11:27–30Google Scholar
  4. Cao LJ, Chua KS, Chong WK et al (2003) A comparison of PCA, KPCA, and ICA for dimensionality reduction in support vector machine. Neurocomputing 55(1–2):321–336Google Scholar
  5. Chen YW, Lin CJ (2006) Combining SVMs with various feature selection strategies. In: Guyon I, Gunn S, Nikravesh M et al (ed) Feature extraction: foundations and applications. Springer, Berlin, pp 315–324CrossRefGoogle Scholar
  6. Corsini P, Lazzerini B, Marcelloni F (2005) A new fuzzy relational clustering algorithm based on the fuzzy C-means algorithm. Soft Comput 9(6):439–447CrossRefGoogle Scholar
  7. Cunningham P (2007) Dimension reduction. Technical Report UCD-CSI-2007-7Google Scholar
  8. Drucker H, Burges CJC, Kaufman H et al (1996) Support vector regression machines. Adv Neural Inf Process Syst 9:155–161Google Scholar
  9. Dy JG, Brodley CE (2004) Feature selection for unsupervised learning. J Mach Learn Res 5:845–889MathSciNetGoogle Scholar
  10. Fodor IK (2002) A survey of dimension reduction techniques. Technical Report UCRL-ID-148494Google Scholar
  11. Gershenfeld N, Weigend A (1994) The Santa Fe time series competition data. http://www-psych.stanford.edu/~andreas/Time-Series/SantaFe.html. Accessed 10 May 2008
  12. Guyon I, Elisseeff A (2003) An introduction to variable and feature selection. J Mach Learn Res 3:1157–1182MATHCrossRefGoogle Scholar
  13. Horn D, Gottlieb A (2001) The method of quantum clustering. Proc Neural Inf Process Syst 14:769–776Google Scholar
  14. Horn D, Gottlieb A (2002) Algorithm for data clustering in pattern recognition problems based on quantum mechanics. Phys Rev Lett 88(1):018702CrossRefGoogle Scholar
  15. Hyvärinen A, Oja E (2000) Independent component analysis: algorithms and applications. Neural Netw 13(4–5):411–430CrossRefGoogle Scholar
  16. Jolliffe IT (2002) Principal component analysis, 2nd edn. Springer, New YorkMATHGoogle Scholar
  17. Mangasarian OL, David R (2002) Large scale kernel regression via linear programming. Mach Learn 46(1–3):255–269MATHCrossRefGoogle Scholar
  18. Pedroso JP, Murata N (1999) Support vector machines for linear programming: motivation and formulations. BSIS Technical Report 99-2, Riken Brain Science Institute, Wako-shi, Saitatma, JapanGoogle Scholar
  19. Reynolds RG (2002) Cultural algorithms: a tutorial. Wayne State University, DetroitGoogle Scholar
  20. Schölkopf B, Smola A (2002) Learning with kernels. MIT Press, CambridgeGoogle Scholar
  21. Schölkopf B, Bartlett P, Smola A et al (1998) Shrinking the tube: a new support vector regression algorithm. Adv Neural Inf Process Syst 11:330–336Google Scholar
  22. Schölkopf B, Smola A, Müller K (1999) Kernel principal component analysis. In: Schölkopf B, Burges C, Smola A (ed) Advances in kernel methods: support vector learning. MIT Press, Cambridge, pp 327–352Google Scholar
  23. Smola AJ, Schölkopf B (2004) A tutorial on support vector regression. Stat Comput 14(3):199–222CrossRefMathSciNetGoogle Scholar
  24. Smola A, Schölkopf B, Rätsch G (1999) Linear programs for automatic accuracy control in regression. Ninth Int Conf Artif Neural Netw 2:575–580CrossRefGoogle Scholar
  25. Suykens JAK, De Brabanter J, Lukas L et al (2000) Weighted least squares support vector machines: robustness and sparse approximation. Neurocomputing 48(1–4):85–105Google Scholar
  26. Suykens JAK, Gestel TV, Vandewalle J et al (2003) A support vector machine formulation to PCA analysis and its kernel version. IEEE Trans Neural Netw 14(2):447–450CrossRefGoogle Scholar
  27. Vapnik V (1995) The nature of statistical learning theory. Springer, BerlinMATHGoogle Scholar
  28. Vapnik V (1998) Statistical learning theory. Wiley, New YorkMATHGoogle Scholar
  29. Vapnik V, Golowich S, Smola A (1996) Support vector method for function approximation, regression estimation, and signal processing. Adv Neural Inf Process Syst 9:281–287Google Scholar
  30. Vlachos P (2005) StatLib—datasets archive. http://lib.stat.cmu.edu/datasets. Accessed 6 Mar 2007
  31. Zheng WM, Zou CR, Zhao L (2005) An improved algorithm for kernel principal component analysis. Neural Process Lett 22(1):49–56CrossRefGoogle Scholar
  32. Zhou W, Zhang L, Jiao L (2002) Linear programming support vector machines. Pattern Recognit 35(12):2927–2936MATHCrossRefGoogle Scholar

Copyright information

© Springer-Verlag 2009

Authors and Affiliations

  1. 1.Key Laboratory of Advanced Control and Optimization for Chemical Processes, Ministry of EducationEast China University of Science and TechnologyShanghaiChina

Personalised recommendations