Unconstrained convex minimization based implicit Lagrangian twin extreme learning machine for classification (ULTELMC)

  • Parashjyoti Borah
  • Deepak GuptaEmail author


The recently proposed twin extreme learning machine (TELM) requires solving two quadratic programming problems (QPPs) in order to find two non-parallel hypersurfaces in the feature that brings in the additional requirement of external optimization toolbox such as MOSEK. In this paper, we propose implicit Lagrangian TELM for classification via unconstrained convex minimization problem (ULTELMC) and further suggest iterative convergent schemes which eliminates the requirement of external optimization toolbox generally required in solving the quadratic programming problems (QPPs) of TELM. The solutions to the dual variables of the proposed ULTELMC are obtained using iterative schemes containing ‘plus’ function which is not differentiable. To overcome this shortcoming, the generalized derivative approach and smooth approximation approaches are suggested. Further, to test the performance of the proposed approaches, classification performances are compared with support vector machine (SVM), twin support vector machine (TWSVM), extreme learning machine (ELM), twin extreme learning machine (TELM) and Lagrangian extreme learning machine (LELM). Moreover, non-requirement to solve QPPs makes the iterative schemes find the solution faster as compared to the reported methods that finds the solution in dual space. Computational times required in finding the solutions are also presented for comparison.


Extreme learning machine Unconstrained minimization Smoothing approaches Quadratic programming problem Iterative schemes 



  1. 1.
    Avci D, Dogantekin A (2016) An Expert diagnosis system for Parkinson disease based on genetic algorithm-wavelet kernel-extreme learning machine. Parkinson’s Dis 2016:5264743Google Scholar
  2. 2.
    Balasundaram S, Gupta D (2014) 1-norm extreme learning machine for regression and multiclass classification using Newton method. Neurocomputing 128:4–14CrossRefGoogle Scholar
  3. 3.
    Balasundaram S, Gupta D (2016) On optimization based extreme learning machine in primal for regression and classification by functional iterative method. Int J Mach Learn Cybern 7(5):707–728CrossRefGoogle Scholar
  4. 4.
    Balasundaram S, Gupta D, Prasad SC (2017) A new approach for training Lagrangian twin support vector machine via unconstrained convex minimization. Appl Intell 46(1):124–134CrossRefGoogle Scholar
  5. 5.
    Bi J, Zhang C (2018) An empirical comparison on state-of-the-art multi-class imbalance learning algorithms and a new diversified ensemble learning scheme. Knowl-Based Syst 158:81–93CrossRefGoogle Scholar
  6. 6.
    Cortes C, Vapnik V (1995) Support-vector networks. Mach Learn 20(3):273–297zbMATHGoogle Scholar
  7. 7.
    Demšar J (2006) Statistical comparisons of classifiers over multiple data sets. J Mach Learn Res 7(Jan):1–30MathSciNetzbMATHGoogle Scholar
  8. 8.
    Deng W, Zheng Q, Chen L (2009, March). Regularized extreme learning machine. In: IEEE symposium on computational intelligence and data mining, 2009. CIDM’09. IEEE, pp 389–395Google Scholar
  9. 9.
    Drucker H, Burges CJ, Kaufman L, Smola AJ, Vapnik V (1997) Support vector regression machines. In: Advances in neural information processing systems, pp 155–161Google Scholar
  10. 10.
    Gupta D, Borah P, Prasad M (2017) A fuzzy based Lagrangian twin parametric-margin support vector machine (FLTPMSVM). In: 2017 IEEE Symposium Series on Computational Intelligence (SSCI), Honolulu, pp 1–7.
  11. 11.
    Gupta D, Richhariya B (2018) Entropy based fuzzy least squares support vector machine for class imbalance learning. Appl Intell 48:4212–4231. CrossRefGoogle Scholar
  12. 12.
    Gupta D, Richhariya B, Borah P (2018) A fuzzy twin support vector machine based on information entropy for class imbalance learning. Neural Comput Applic 31:7153–7164. CrossRefGoogle Scholar
  13. 13.
    Huang GB, Chen L (2007) Convex incremental extreme learning machine. Neurocomputing 70(16–18):3056–3062CrossRefGoogle Scholar
  14. 14.
    Huang GB, Ding X, Zhou H (2010) Optimization method based extreme learning machine for classification. Neurocomputing 74(1–3):155–163CrossRefGoogle Scholar
  15. 15.
    Huang GB, Wang DH, Lan Y (2011) Extreme learning machines: a survey. Int J Mach Learn Cybern 2(2):107–122CrossRefGoogle Scholar
  16. 16.
    Huang GB, Zhu QY, Siew CK (2004, July) Extreme learning machine: a new learning scheme of feedforward neural networks. In: Proceedings of 2004 IEEE international joint conference on neural networks, 2004, vol 2. IEEE, pp 985–990Google Scholar
  17. 17.
    Ismaeel S, Miri A, Chourishi D (2015, May) Using the extreme learning machine (ELM) technique for heart disease diagnosis. In: Humanitarian Technology Conference (IHTC2015), 2015 IEEE Canada International. IEEE, pp 1–3Google Scholar
  18. 18.
    Jayadeva, Khemchandani R, Chandra S (2007) Twin support vector machines for pattern classification. IEEE Trans Pattern Anal Mach Intell 29(5):905–910CrossRefGoogle Scholar
  19. 19.
    Joachims T (1999) Making large-scale SVM learning practical. In: Schölkopf B, Burges C, Smola A (eds) Advances in kernel methods – support vector learning. MIT Press, CambridgeGoogle Scholar
  20. 20.
    Lee YJ, Mangasarian OL (2001) SSVM: a smooth support vector machine for classification. Comput Optim Appl 20(1):5–22MathSciNetCrossRefGoogle Scholar
  21. 21.
    Li Q, Chen H, Huang H, Zhao X, Cai Z, Tong C et al (2017) An enhanced grey wolf optimization based feature selection wrapped kernel extreme learning machine for medical diagnosis. Comput Math Methods Med 2017:9512741Google Scholar
  22. 22.
    Ma J, Wen Y, Yang L (2019) Lagrangian supervised and semi-supervised extreme learning machine. Appl Intell 49(2):303–318CrossRefGoogle Scholar
  23. 23.
    Mangasarian OL, Musicant DR (2001) Lagrangian support vector machines. J Mach Learn Res 1(Mar):161–177MathSciNetzbMATHGoogle Scholar
  24. 24.
    Mangasarian OL (2004) A Newton method for linear programming. J Optim Theory Appl 121:1–18 MathSciNetCrossRefGoogle Scholar
  25. 25.
    Miche Y, Sorjamaa A, Bas P, Simula O, Jutten C, Lendasse A (2010) OP-ELM: optimally pruned extreme learning machine. IEEE Trans Neural Netw 21(1):158–162CrossRefGoogle Scholar
  26. 26.
    Mitra SK, Rao CR (1971) Generalized inverse of matrices and its applications. Wiley, New YorkzbMATHGoogle Scholar
  27. 27.
    Musicant DR, Feinberg A (2004) Active set support vector regression. IEEE Trans Neural Netw 15(2):268–275CrossRefGoogle Scholar
  28. 28.
    Murphy PM, Aha DW (1992) UCI repository of machine learning databases. Department of Information and Computer Science, University of California, IrvineGoogle Scholar
  29. 29.
    Muthusamy H, Polat K, Yaacob S (2015) Improved emotion recognition using gaussian mixture model and extreme learning machine in speech and glottal signals. Math Probl Eng 2015:394083CrossRefGoogle Scholar
  30. 30.
    Ning K, Liu M, Dong M, Wu C, Wu Z (2015) Two efficient twin ELM methods with prediction interval. IEEE Trans Neural Netw Learn Syst 26(9):2058–2071MathSciNetCrossRefGoogle Scholar
  31. 31.
    Peng X (2010) Primal twin support vector regression and its sparse approximation. Neurocomputing 73:2846–2858CrossRefGoogle Scholar
  32. 32.
    Peng Y, Wang S, Long X, Lu BL (2015) Discriminative graph regularized extreme learning machine and its application to face recognition. Neurocomputing 149:340–353CrossRefGoogle Scholar
  33. 33.
    Rastogi R, Sharma S, Chandra S (2018) Robust parametric twin support vector machine for pattern classification. Neural Process Lett 47(1):293–323CrossRefGoogle Scholar
  34. 34.
    Richhariya B, Tanveer M (2018) A robust fuzzy least squares twin support vector machine for class imbalance learning. Appl Soft Comput Elsevier 71:418–432CrossRefGoogle Scholar
  35. 35.
    Ripley BD (2007) Pattern recognition and neural networks. Cambridge university press, CambridgezbMATHGoogle Scholar
  36. 36.
    Rozza A, Manzo M, Petrosino A (2014, August) A novel graph-based fisher kernel method for semi-supervised learning. In: 2014 22nd International Conference on Pattern Recognition (ICPR). IEEE, pp 3786–3791Google Scholar
  37. 37.
    Suykens JA, Vandewalle J (1999) Least squares support vector machine classifiers. Neural Process Lett 9(3):293–300CrossRefGoogle Scholar
  38. 38.
    Uçar A, Demir Y, Güzeliş C (2016) A new facial expression recognition based on curvelet transform and online sequential extreme learning machine initialized with spherical clustering. Neural Comput Applic 27(1):131–142CrossRefGoogle Scholar
  39. 39.
    Wan Y, Song S, Huang G, Li S (2017) Twin extreme learning machines for pattern classification. Neurocomputing 260:235–244CrossRefGoogle Scholar
  40. 40.
    Xue Z, Zhang R, Qin C, Zeng X (2018) A rough ν-twin support vector regression machine. Appl Intell 48:1–24CrossRefGoogle Scholar
  41. 41.
    Zhou F, Yang S, Fujita H, Chen D, Wen C (2019) Deep learning fault diagnosis method based on global optimization GAN for unbalanced data. Knowl-Based SystGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2020

Authors and Affiliations

  1. 1.Department of Computer Science & EngineeringNational Institute of TechnologyYupiaIndia

Personalised recommendations