Twin support vector regression for the simultaneous learning of a function and its derivatives

  • Reshma Khemchandani
  • Anuj Karpatne
  • Suresh Chandra
Original Article


Twin support vector regression (TSVR) determines a pair of \(\epsilon\)-insensitive up- and down-bound functions by solving two related support vector machine-type problems, each of which is smaller than that in a classical SVR. On the lines of TSVR, we have proposed a novel regressor for the simultaneous learning of a function and its derivatives, termed as TSVR of a Function and its Derivatives. Results over several functions of more than one variable demonstrate its effectiveness over other existing approaches in terms of improving the estimation accuracy and reducing run time complexity.


Twin support vector machines Support vector regression \(\epsilon\)-insensitive bound Function approximation 


  1. 1.
    Antonio J, Martìn H, Santos M, Lope J (2010) Orthogonal variant moments features in image analysis. Inform Sci 180:846–860MathSciNetCrossRefGoogle Scholar
  2. 2.
    Burges C (1998) A tutorial on support vector machines for pattern recognition. Data Min Knowl Discov 2(2):121–167CrossRefGoogle Scholar
  3. 3.
    Beauchemin SS, Barron JL (1995) The computation of optical flow. ACM, New YorkGoogle Scholar
  4. 4.
    Chen C, Zhang J, He X, Zhou Z (2011) Non-parametric Kernel learning with robust pairwise constraints. Int J Mach Learn Cybern. doi: 10.1007/s13042-011-0048-6
  5. 5.
    Christianini N, Shawe-Taylor J (2000) An intorduction to support vector machines. Cambridge University Press, CambridgeGoogle Scholar
  6. 6.
    Ebrahimi T, Garcia G, Vesin J (2002) Joint time-frequency-space classification of EEG in a brain–computer interface appplication. J Apply Signal Process 1:713–729Google Scholar
  7. 7.
    Fung G, Mangasarian OL (2001) Incremental support vector machine classification. In: 7th ACM SIGKDD international conference on knowledge discovery and data mining, pp 77–86Google Scholar
  8. 8.
    Golub GH, Van Loan CF (1996) Matrix computations, 3rd edn. The John Hopkins Univ. Press, MarylandGoogle Scholar
  9. 9.
    He F, IAENG M, Yeung LM, Brown M (2008) Discrete-Time Model Representation for Biochemical Pathway Systems. IAENG Int J Comput Sci 34(1):1–15Google Scholar
  10. 10.
    Ince H, Trafalis TB (2000) Support vector machine for regression and applications to financial forecasting, In: International joint conference on neural networks, IJCNN, vol 6, pp 348–353Google Scholar
  11. 11.
    Jayadeva, Khemchandani R, Chandra S (2006) Regularized least squares twin SVR for the simultaneous learning of a function and its derivative, IJCNN, pp 1192–1197Google Scholar
  12. 12.
    Jayadeva, Khemchandani R, Chandra S (2007) Twin support vector machines for pattern classification. IEEE Trans Pattern Anal Mach Intell 29:905–910Google Scholar
  13. 13.
    Jayadeva, Khemchandani R, Chandra S (2008) Regularized least squares support vector regression for the simultaneous learning of a function and its derivatives. Inform Sci 178:3402–3414Google Scholar
  14. 14.
    Joachims T (1999) Making large-scale SVM learning practical. In: Advances in kernel methods: support vector learning. MIT Press, CambridgeGoogle Scholar
  15. 15.
    Julier SJ, Uhlmann JK (2004) Unscented filtering and nonlinear estimation. In: Proceedings of the IEEE, pp 401–422Google Scholar
  16. 16.
    Khemchandani R, Jayadeva, Chandra S (2009) Regularized least squares fuzzy support vector regression for financial time series forecasting. Exp Syst Appl 36:132–138Google Scholar
  17. 17.
    Lagaris IE, Likas A, Fotiadis D (1998) Artificial neural networks for solving ordinary and partial differential equations. IEEE Trans Neural Netw 9:987–1000CrossRefGoogle Scholar
  18. 18.
    Lengagne R (2000) 3D stereo reconstruction of human faces driven by differential constraints. Image Vis Comput 18:337-343CrossRefGoogle Scholar
  19. 19.
    Lázaro M, Santamaŕia I, Pérez-Cruz F, Artés-Rodŕiguez A (2005) Support vector regression for the simultaneous learning of a multivariate function and its derivative. Neurocomputing 69:42–61CrossRefGoogle Scholar
  20. 20.
    Lázaro M, Santamaria I, Pérez-Cruz F, Artés-Rodriguez A (2003) SVM for the simultaneous approximation of a function and its derivative. In: Proceedings of the 2003 IEEE international workshop on neural networks for signal processing (NNSP), Toulouse, France, pp 189–198Google Scholar
  21. 21.
    Mangasarian OL (1998) Nonlinear programming. SIAMGoogle Scholar
  22. 22.
    Liu Z, Wu Q, Zhang Y, Chen CLP (2011) Adaptive least squares support vector machines filter for hand tremor canceling in microsurgery. Int J Mach Learn Cybern 2(1):37–47MathSciNetCrossRefGoogle Scholar
  23. 23.
    Mees AI, Jackson MF, Chua LO (1992) Device modeling by radial basis function. IEEE Trans Circuits Syst I Fundam Theory Appl 39:19–27CrossRefGoogle Scholar
  24. 24.
    Osuna E, Freund R, Girosi F (1997) Training support vector machines: an application to face detection. In: IEEE computer society conference on computer vision and pattern recognition, pp 130–136Google Scholar
  25. 25.
    Peng X (2009) TSVR: an efficient twin support vector machine for regression. Neural Netw 23:365–372CrossRefGoogle Scholar
  26. 26.
    Pérez-Cruz F, Bousono-Calzón C, Artés-Rodriguez A (2005) Convergence of the IRWLS procedure to the support vector machine solution. Neural Comput 17:7–18zbMATHCrossRefGoogle Scholar
  27. 27.
    Schmidt M, Lipson H (2009) Distilling free-form natural laws from experimental data. Science 324(5923):81–85CrossRefGoogle Scholar
  28. 28.
    Kemelmacher-Shlizerman I, Basri R (2011) 3D face reconstruction from a single image using a single reference face shape. IEEE Trans Pattern Anal Mach Intell 33(2):394–405CrossRefGoogle Scholar
  29. 29.
    Vapnik VN (1995) The nature of statistical learning theory. Springer, BerlinGoogle Scholar
  30. 30.
    Vapnik VN (1998) Statistical learning theory. Wiley, NYGoogle Scholar
  31. 31.
    Wang Z, Lu S, Zhai J (2008) Fast fuzzy multi-category SVM based on support vector domain description. Int J Pattern Recogn Artif Intell 22(1):109–120CrossRefGoogle Scholar
  32. 32.
    Xiao JZ, Wang HR, Yang X, Gao Z (2011) Multiple faults diagnosis in motion system based on SVM. Int J Mach Learn Cybern. doi: 10.1007/s13042-011-0035-y
  33. 33.
    Zheng S (2011) Gradient descent algorithms for quantile regression with smooth approximation. Int J Mach Learn Cybern 2(3):191–207CrossRefGoogle Scholar

Copyright information

© Springer-Verlag 2012

Authors and Affiliations

  • Reshma Khemchandani
    • 1
  • Anuj Karpatne
    • 2
  • Suresh Chandra
    • 2
  1. 1.RBS India Development CentreGurgaonIndia
  2. 2.Department of MathematicsIndian Institute of TechnologyNew DelhiIndia

Personalised recommendations