Advertisement

Applied Intelligence

, Volume 45, Issue 4, pp 1148–1165 | Cite as

Novel hybrid SVM-TLBO forecasting model incorporating dimensionality reduction techniques

  • Shom Prasad DasEmail author
  • N. Sangita Achary
  • Sudarsan Padhy
Article

Abstract

In this paper, we present a highly accurate forecasting method that supports improved investment decisions. The proposed method extends the novel hybrid SVM-TLBO model consisting of a support vector machine (SVM) and a teaching-learning-based optimization (TLBO) method that determines the optimal SVM parameters, by combining it with dimensional reduction techniques (DR-SVM-TLBO). The dimension reduction techniques (feature extraction approach) extract critical, non-collinear, relevant, and de-noised information from the input variables (features), and reduce the time complexity. We investigated three different feature extraction techniques: principal component analysis, kernel principal component analysis, and independent component analysis. The feasibility and effectiveness of this proposed ensemble model were examined using a case study, predicting the daily closing prices of the COMDEX commodity futures index traded in the Multi Commodity Exchange of India Limited. In this study, we assessed the performance of the new ensemble model with the three feature extraction techniques, using different performance metrics and statistical measures. We compared our results with results from a standard SVM model and an SVM-TLBO hybrid model. Our experimental results show that the new ensemble model is viable and effective, and provides better predictions. This proposed model can provide technical support for better financial investment decisions and can be used as an alternative model for forecasting tasks that require more accurate predictions.

Keywords

Support Vector Machine (SVM) Teaching-Learning-Based Optimization (TLBO) Dimensional reduction Commodity futures contract Financial time series 

Notes

Acknowledgments

We would like to express our gratitude to the National Institute of Science and Technology (NIST), for the facilities and resources provided at the Data Science Laboratory at NIST for the development of this study.

Compliance with ethical standards

Conflict of interests

The authors declare that there are no conflict of interests (either financial or non-financial) regarding the publication of the paper.

References

  1. 1.
    Cai LJ, Zhang JQ, Zongwu CAI, Kian Guan LIM (2006) An empirical study of dimensionality reduction in support vector machine. Neural Network World 16(3):177–192Google Scholar
  2. 2.
    Cao LJ (2003) Support vector machines experts for time series forecasting. Neorocomputing 51:321–339CrossRefGoogle Scholar
  3. 3.
    Cao LJ, Chua KS, Chong WK, Lee HP, Gu QM (2003) A comparison of PCA, KPCA and ICA for dimensional reduction in support vector machines. Neurocomputing 55(1):321–336Google Scholar
  4. 4.
    Cao LJ, Tay FEH (2003) Support vector machine with adaptive parameters in financial time series forecasting. IEEE Trans Neural Netw 14(6):1506–1518CrossRefGoogle Scholar
  5. 5.
    Chang CC, Lin CJ (2011) LIBSVM: A library for support vector machines. ACM Trans Intell Syst Technol (TIST) 2(3):27Google Scholar
  6. 6.
    Chang PC, Wu JL (2015) A critical feature extraction by kernel PCA in stock trading model. Soft Comput 19(5):1393–1408CrossRefGoogle Scholar
  7. 7.
    Chen WH, Shih JY, Wu S (2006) Comparison of support-vector machines and back propagation neural networks in forecasting the six major Asian stock markets. International Journal of Electronic Finance 1(1):49–67CrossRefGoogle Scholar
  8. 8.
    Das SP, Padhy S (2015) A novel hybrid model using teaching–learning-based optimization and a support vector machine for commodity futures index forecasting. Int J Mach Learn Cyber:1–15. doi: 10.1007/s13042-015-0359-0
  9. 9.
    Diebold FX, Mariano RS (1995) Comparing predictive accuracy. J Bus Econ Stat 13(3):253–263MathSciNetGoogle Scholar
  10. 10.
    Ekenel HK Sankur B (2004) Feature selection in the independent component subspace for face recognition. Pattern Recogn Lett 25(12):377–1388Google Scholar
  11. 11.
    Haykin S (2010) Neural Networks and Learning Machines. 3rd Edition, PHI Learning Private LimitedGoogle Scholar
  12. 12.
    Hsu CM (2013) A hybrid procedure with feature selection for resolving stock/futures price forecasting problems. Neural Comput Applic 22(3–4):651–671. doi: 10.1007/s00521-011-07214 CrossRefGoogle Scholar
  13. 13.
    Huang CL, Tsai CY (2009) A hybrid SOFM-SVR with a filter based feature selection for stock market forecasting. Expert Syst Appl 36(2):1529–1539. doi: 10.1016/j.eswa.2007.11.062 MathSciNetCrossRefGoogle Scholar
  14. 14.
    Hyvarinen A, Karhunen J, Oja E (2001) Independent component analysis. Wiley, New YorkCrossRefGoogle Scholar
  15. 15.
    Hyvarinen A, Oja E (1997) A fast fixed-point algorithm for independent component analysis. Neural Comput 9(7):1483– 1492CrossRefGoogle Scholar
  16. 16.
    Hyvarinen A, Oja E (2000) Independent component analysis: algorithms and applications. Neural networks 13(4):411–430CrossRefGoogle Scholar
  17. 17.
    Ince H, Trafalis TB (2007) Kernel principal component analysis and support vector machines for stock price prediction. IIE Trans 39(6):629–637CrossRefGoogle Scholar
  18. 18.
    Ince H, Trafalis TB (2008) Short term forecasting with support vector machines and application to stock price prediction. Int J Gen Syst 37(6):77–687. doi: 10.1080/03081070601068595 MathSciNetCrossRefzbMATHGoogle Scholar
  19. 19.
    Jiang M, Jiang S, Zhu L, Wang Y, Huang W, Zhang H (2013) Study on parameter optimization for support vector regression in solving the inverse ECG problem. Comput Math Methods Med Article ID 158059. doi: 10.1155/2013/158056 MathSciNetzbMATHGoogle Scholar
  20. 20.
    Jolliffe IT (2002) Principle components analysis 2 nd Edition. Springer, New YorkGoogle Scholar
  21. 21.
    Kim KJ (2003) Financial time series forecasting using support vector machines. Neurocomputing 55(1):307–319CrossRefGoogle Scholar
  22. 22.
    Kim KJ, Han I (2000) Genetic algorithms approach to feature discretization in artificial neural networks for the prediction of stock price index. Expert Syst Appl 19(2):125–132MathSciNetCrossRefGoogle Scholar
  23. 23.
    Kim KJ, Lee WB (2004) Stock market prediction using artificial neural networks with optimal feature transformation. Neural Comput Applic 13(3):255–260. doi: 10.1007/s00521-004-0428-x CrossRefGoogle Scholar
  24. 24.
    Kuang F, Zhang S, Jin Z, Xu W (2015) A novel SVM by combining kernel principal component analysis and improved chaotic particle swarm optimization for intrusion detection. Soft Comput 19:1187–1199. doi: 10.1007/s00500-014-1332-7 CrossRefGoogle Scholar
  25. 25.
    Lai RK, Fan CY, Huang WH, Chang PC (2009) Evolving and clustering fuzzy decision tree for financial time series data forecasting. Expert Syst Appl 36(2):3761–3773. doi: 10.1016/j.eswa.2008.02.025 CrossRefGoogle Scholar
  26. 26.
    Leung MT, Daouk H, Chen AS (2000) Forecasting stock indices: a comparison of classification and level estimation models. Int J Forecast 16(2):173–190CrossRefGoogle Scholar
  27. 27.
    Liang X, Zhang H, Xiao J, Chen Y (2009) Improving option price forecasts with neural networks and support vector regressions. Neurocomputing 72(13):3055–3065. doi: 10.1016/j.neucom.2009.03.015 CrossRefGoogle Scholar
  28. 28.
    Lin HT, Lin CJ (2003) A study on sigmoid kernels for SVM and the training of non-PSD kernels by SMO-type methods Technical report, University of National Taiwan Department of Computer Science and Information Engineering, March 1–32Google Scholar
  29. 29.
    Lin SW, Ying KC, Chen SC, Lee ZJ (2008) Particle swarm optimization for parameter determination and feature selection of support vector machines. Expert Syst Appl 35(4):1817– 1824CrossRefGoogle Scholar
  30. 30.
    Liu S, Tian L, Huang Y (2014) A comparative study on prediction of throughput in coal ports among three models. Int J Mach Learn Cybern 5(1):125–133. doi: 10.1007/s13042-013-0201-5 CrossRefGoogle Scholar
  31. 31.
    Lu CJ (2013) Hybridizing nonlinear independent component analysis and support vector regression with particle swarm optimization for stock index forecasting. Neural Comput Applic 23(7–8):2417–2427. doi: 10.1007/s00521-012-1198-5 CrossRefGoogle Scholar
  32. 32.
    Lu CJ, Lee TS, Chiu CC (2009) Financial time series forecasting using independent component analysis and support vector regression. Decis Support Syst 47(2):115–125CrossRefGoogle Scholar
  33. 33.
    Musa AB (2014) A comparison of 1-regularizion, PCA, KPCA and ICA for dimensionality reduction in logistic regression. Int J Mach Learn Cybern 5(6):861–873. doi: 10.1007/s13042-013-0171-7 CrossRefGoogle Scholar
  34. 34.
    Pawar PV, Rao RV (2013) Parameter optimization of machining using teaching-learning-based optimization algorithm. Int J Adv Manuf Technol 67:995–1006CrossRefGoogle Scholar
  35. 35.
    Porikli F, Haga T (2004) Event detection by eigenvector decomposition using object and frame features. IEEE Conference In Computer Vision and Pattern Recognition Workshop 2004(CVPRW’04):114–114Google Scholar
  36. 36.
    Rao RV, Patel V (2014) A multi-objective improved teaching-learning based optimization algorithm for unconstrained and constrained optimization problems. Int J Ind Eng Comput 5(1):1–22. doi: 10.5267/j.ijiec.2013.09.007 Google Scholar
  37. 37.
    Rao RV, Savsani VJ, Vakharia DP (2011) Teaching-learning-based optimization: a novel method for constrained mechanical design optimization problems. Comput Aided Des 43(3):303– 315CrossRefGoogle Scholar
  38. 38.
    Sapankevych NI, Sankar R (2009) Time series prediction using support vector machines: a survey. IEEE Comput Intell Mag 4(2):24–38. doi: 10.1109/MCI.2009.932254 CrossRefGoogle Scholar
  39. 39.
    Tay FE, Cao LJ (2002) Modified support vector machines in financial time series forecasting. Neurocomputing 48(1):847– 861CrossRefzbMATHGoogle Scholar
  40. 40.
    Tsai CF, Hsiao YC (2010) Combining multiple feature selection methods for stock prediction: Union, intersection, and multi-intersection approaches. Decis Support Syst 50(1):258–269. doi: 10.1007/s00500-014-1350-5 CrossRefGoogle Scholar
  41. 41.
    Tsang PM, Kwok P, Choy SO, Kwan R, Ng SC, Mak J, Tsang J, Koong K, Wong TL (2007) Design and implementation of NN5 for Hong Kong stock price forecasting. Eng Appl Artif Intell 20 (4):453–461. doi: 10.1016/j.engappai.2006.10.002 CrossRefGoogle Scholar
  42. 42.
    Twining CJ, Taylor CJ (2003) The use of kernel principal component analysis to model data distributions. Pattern Recogn 36(1):217–227CrossRefzbMATHGoogle Scholar
  43. 43.
    Van Gestel T, Suykens JA, Baestaens DE, Lambrechts A, Lanckriet G, Vandaele B, Vandewalle J (2001) Financial time series prediction using least squares support vector machines within the evidence framework. IEEE Trans Neural Netw 12(4):809–821CrossRefGoogle Scholar
  44. 44.
    Vapnik V (1995) The nature of statistical learning theory. Springer, NYCrossRefzbMATHGoogle Scholar
  45. 45.
    Wang J, Wang J (2015) Forecasting stock market indexes using principle component analysis and stochastic time effective neural networks. Neurocomputing 156:68–78CrossRefGoogle Scholar
  46. 46.
    Wang S, Meng B (2011) Parameter selection algorithm for support vector machine. Prog Environ Sci 11:538–544. doi: 10.1016/j.proenv.2011.12.085 CrossRefGoogle Scholar
  47. 47.
    Wu CH, Tzeng GH, Lin RH (2009) A Novel hybrid genetic algorithm for kernel function and parameter optimization in support vector regression. Expert Syst Appl 36(3):4725–4735. doi: 10.1016/j.eswa.2008.06.046 CrossRefGoogle Scholar
  48. 48.
    Zhai G, Chen J, Wang S, Li K, Zhang L (2015) Material identification of loose particles in sealed electronics devices using PCA and SVM. Neurocomputing 148:222–228. doi: 10.1016/j.neucom.2013.10.043 CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2016

Authors and Affiliations

  • Shom Prasad Das
    • 1
    Email author
  • N. Sangita Achary
    • 1
  • Sudarsan Padhy
    • 2
  1. 1.Department of Computer Science and EngineeringNational Institute of Science and TechnologyPalur HillsIndia
  2. 2.Silicon Institute of TechnologySilicon Hills, BhubaneswarIndia

Personalised recommendations