Proximal support tensor machines

  • Reshma Khemchandani
  • Anuj Karpatne
  • Suresh Chandra
Original Article


To utilize the structural information present in multidimensional features of an object, a tensor-based learning framework, termed as support tensor machines (STMs), was developed on the lines of support vector machines. In order to improve it further we have developed a least squares variant of STM, termed as proximal support tensor machine (PSTM), where the classifier is obtained by solving a system of linear equations rather than a quadratic programming problem at each iteration of PSTM algorithm as compared to STM algorithm. This in turn provides a significant reduction in the computation time, as well as comparable classification accuracy. The efficacy of the proposed method has been demonstrated in simulations over face detection and handwriting recognition datasets.


Support tensor machines Alternating projection Proximal support vector machines 


  1. AT&T Labs Cambridge (1994) The Olivetti and Oracle Research Laboratory database of faces.
  2. Alpaydin E, Kaynak C (1998) UCI machine learning repository. University of California, School of Information and Computer Science, Irvine.
  3. Cai D, He X, Wen J-R, Han J, Ma W-Y (2006) Support tensor machines for text categorization. Technical report, Department of Computer Science, UIUC. UIUCDCS-R-2006-2714Google Scholar
  4. Cristianini N, Shawe-Taylor J (2000) An introduction to support vector machines: and other kernel-based learning methods. Cambridge University Press, New YorkGoogle Scholar
  5. Duda R, Hart P, Stork D (2001) Pattern classification. Wiley, New YorkGoogle Scholar
  6. Fu Y, Huang TS (2008) Image classification using correlation tensor analysis. IEEE Trans Image Process 17(2):226–234MathSciNetCrossRefGoogle Scholar
  7. Fung G, Mangasarian OL (2001) Proximal support vector machine classifiers. In: KDD-2001: knowledge discovery and data mining, San Francisco, pp 77–86.Google Scholar
  8. Golub GH, Van Loan CF (1996) Matrix computations. 3rd ed. The John Hopkins University Press, BaltimoreGoogle Scholar
  9. Guo G, Li SZ, Chan K (2000) Face recognition by support vector machines. In: IEEE international conference on automatic face and gesture recognition, Grenoble, pp 196–201.Google Scholar
  10. He X, Cai D, Niyogi P (2005) Tensor subspace analysis. In: Advances in neural information processing systems. Vancouver, CanadaGoogle Scholar
  11. Hsu CW, Lin CJ (2002) A comparison of methods for multiclass support vector machines. IEEE Trans Neural Netw 13(2):415–425CrossRefGoogle Scholar
  12. Hu RX, Jia W, Huang DS, Lei YK (2010) Maximum margin criterion with tensor representation. Neurocomputing 73(10–12):1541–1549CrossRefGoogle Scholar
  13. Jayadeva, Khemchandani R, Chandra S (2007) Twin support vector machines for pattern classification. IEEE Trans Pattern Anal Mach Intell 29(5):905–910Google Scholar
  14. Jonsson K, Matas J, Kittler J, Li YP (2002) Learning support vectors for face verification and recognition. In: Proceeding of the IEEE international conference on automatic face and gesture recognition, Grenoble, pp 208–213Google Scholar
  15. Liu Z, Wu Q, Zhang Y, Chen CL (2011) Adaptive least squares support vector machines filter for hand tremor canceling in microsurgery. Int J Mach Learn Cybern 2(1):37–47MathSciNetCrossRefGoogle Scholar
  16. Plamondon R, Srihari SN (2000) On-line and off-line handwriting recognition: a comprehensive survey. IEEE Trans Pattern Anal Mach Intell 22:63–84CrossRefGoogle Scholar
  17. Maulik U, Chakraborty D (2011) A novel semisupervised SVM for pixel classification of remote sensing imagery. Int J Mach Learn Cybern. doi: 10.1007/s13042-011-0059-3
  18. Mangasarian OL (1994) Nonlinear programming, SIAMGoogle Scholar
  19. Kumar MA, Gopal M (2009) Least squares twin support vector machines for pattern classification. Expert Syst Appl 36(4):7535–7543CrossRefGoogle Scholar
  20. Sun J, Tao D, Papadimitriou S, Yu P, Faloutsos C (2008) Incremental tensor analysis: theory and applications. ACM transactions on knowledge discovery from data (TKDD) TKDD, vol 2, no 3. Article no 11.Google Scholar
  21. Tao D, Li X, Hu W, Maybank SJ, Wu X (2007) General tensor discriminant analysis and Gabor features for gait recognition. IEEE Trans Pattern Anal Mach Intell 29(10):1700–1715CrossRefGoogle Scholar
  22. Tao D, Li X, Hu W, Maybank SJ, Wu X (2008) Tensor rank one discriminant analysis: a convergent method for discriminative multilinear subspace selection. Neurocomputing 71(10–12):1866–1882CrossRefGoogle Scholar
  23. Tao D, Song M, Li X, Shen J, Sun J, Wu X, Faloutsos C, Maybank SJ (2008) Bayesian tensor approach for 3-D face modeling. IEEE Trans Circuits Syst Video Technol 18(10):1397–1410CrossRefGoogle Scholar
  24. Tao D, Sun J, Shen J, Wu X, Li X, Maybank SJ, Faloutsos C (2008) Bayesion tensor analysis. In: IEEE international joint conference on date of conference neural networks, vol 1–8, pp 1402–1409.Google Scholar
  25. Tao D, Li X, Hu W, Maybank SJ, Wu X (2005) Supervised tensor learning. In: ICDM 2005: proceedings of the 5th IEEE international conference on data mining, pp 450–457Google Scholar
  26. Tao D, Li X, Hu W, Maybank SJ, Wu X (2007) Supervised tensor learning. Knowl Inf Syst 13(1):1–42CrossRefGoogle Scholar
  27. Urszula MK, Pawel K (2005) Support vector machines in handwritten digits classification. In: 5th international conference on intelligent systems design and applications (ISDA’05), pp 406–411Google Scholar
  28. Vapnik V (1998) Statistical learning theory. Wiley, New YorkGoogle Scholar
  29. Xiao JZ, Wang HR, Yang YC, Gao Z (2012) Multiple faults diagnosis in motion system based on SVM. Int J Mach Learn Cybern 3(1):77–82CrossRefGoogle Scholar
  30. Xu L, Krzyzak A, Suen CY (1992) Methods of combining multiple classifiers and their applications to handwriting recognition. IEEE Trans Syst Man Cybern 22(3):418–435CrossRefGoogle Scholar
  31. Yan S, Xu D, Yang Q, Zhang L, Tang X, Zhang H (2007) Multilinear discriminant analysis for face recognition. IEEE Trans Image Process 16(1):212–220MathSciNetCrossRefGoogle Scholar
  32. Wang Z, Chen SC, Liu J, Zhang DQ (2008) Pattern representation in feature extraction and classifier design: matrix versus vector. IEEE Trans Neural Netw 19(5):758–769CrossRefGoogle Scholar
  33. Zhang Z, Ye N (2011) Learning a tensor subspace for semi-supervised dimensionality reduction. Soft Comput 15(2):383–395MathSciNetCrossRefzbMATHGoogle Scholar

Copyright information

© Springer-Verlag 2012

Authors and Affiliations

  • Reshma Khemchandani
    • 1
  • Anuj Karpatne
    • 2
  • Suresh Chandra
    • 2
  1. 1.RBS India Development CentreGurgaonIndia
  2. 2.Department of MathematicsIndian Institute of TechnologyHauz KhasIndia

Personalised recommendations